CONTRIBUTING AUTHOR) 


Robert C. Angell 
A. Angus Campbell 
Charles F. Cannell 
Doru'in Tr. Cartiuright 
Clyde H. Coombs 
Leon Festinger 
Ronald Freedman 
John R. P. French, Jr. 
Roger ir. Heyns 
Robert /^. Kahn 
George Katona 
Daniel Katz 
Leslie Kish 
Rensis Likert 
Lippitt 

Theodore M. Newcomb 
Helen Peak 
Keith Smith 
Alvin F. Zander 



RESEARCH METHODS 


IN THE 


BEHAVIORAL SCIENCES 


MLSU- CENTRAL LIBRARY 



EDITED BY 

LEON FESTINGER 

Department of Psychology, Unwersily of Minnesota 


DANIEL KATZ 

Department of Psychology, University of Michigan 


HOLT, RINEHART AND WINSTON 

NEW YORK-CHICAGO-SAN FRANCISCO- 
TORONTO-LONDON 


naio 


Dccemlrer, 196& 

Copyright 1953 by Holtf Rinehart and Winston, Jnc. 

AU rights reserved 
2034153 


Printed in the United States of America 
IS Ifi 17 18 19 



Foreword 


The discovery in our time that scientific methodology can 
be applied to human problems has revolutionized psychology and 
has seriously afEected all branches of social science. This discovery, 
moreover, came during a period when problems of social adjust- 
ment had reached a critical point in the years of depression, of 
war, and of postwar crises. As a consequence, empirical and quanti- 
tative research in our field has seen unparalleled growth. This 
period of boom has naturally not been characterized by a high 
degree of order or of systematic development of theory and meth- 
odology, We have been too absorbed in doing research to plan 
thoroughly, to take stock of our progress, or to communicate our 
finding adeqaacely and to inlocm one another ol onr fecftnfqcfer 
and approaches. 

The first great break in this pattern came with the publication 
of Studies in Social Psychology in World War II, Stouffer and his 
collaborators took time out to set forth their findings and their 
methods in communicable 'form. These volumes were an excellent 
demonstration of the importance of the codification of research 
methods. In the early days of the social studies, there was justifica- 
tion for scholars to give the result of their insights and reflections 
without specifications concerning the -ways in which they arrived 
at their interpretations, for in that period they were working more 
as intuitive artists than as scientists. But today, when we attempt 
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expenmentation and quanlification we have no e-ccusc for failing 
to codify our procedures 

One essential aspect of scientific technique is th it it can ne 
Slated m a standard form and can be taught so tint trained and 
competent investigators can apply it m the same fashion We still 
have not achieved the degree of specification possible in the pliysical 
sciences The method of the interview for example still combines 
art and science It is only however through making our procedures 
explicit that we can lest cniiciie and improve them 

Psychologists social psichologists and sociologists at the Uni 
versity of Michigan have felt fortunate in the favorable atmosphere 
for social research at their institution which has made possible many 
projects m the academic departments and the creation of the 
Institute for Social Research with its coordinate divisions of the 
Survey Research Center and the Research Center for Group Dy 
namics Since this research development brought together many 
specialists ii seemed ivonh while to take advantage of iheir physical 
and psychological proximity to produce a book on methodology 
Two purposes were dominant (1) to help in the present trend 
toward codiflcaUQu of research technw^ues and (2) to give graduate 
students in the field some understanding of the principles and pro 
cedures of modem methodology The criterion for inclusion of meth 
ods was the degree of relevance to the problems of social psychology 
and the criterion for exclusion was the availability of knowledge 
about a technique already standardized in another field Thus al 
though factor analysis is a useful method m social science the details 
of Its application have ahead) been described in statistical texts 
Similarly projective methods have been described in the personality 
context m which they are cliaiacteristically used On the other 
hand there has been a lack of detailed treatment of behavioral 
observation of the quantitative analysis of qualitative materials 
and of such major researcli settings as field studies and field expen 
ments ‘ 


There has been anothci miderl)ing purpose in the publication 
oF this book U vs our beliel that prog:ress in any field must rest 
upon methods appiopriatc to that field Although the basic logic 
o scientific methodology ,s the same in all fields tts specific tech 

3ter ““ ''“y depending upon the subject 

matter In ns early stages socal psychology was handicapped by a 
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lick of methods appioprnte to its pioblcms In general, the recruits 
from the upper frontier of social science understood its larger 
problems but ^vere unequipped as technicians to handle them The 
technology came from tlie lower frontier of individual psychology, 
where there had been a long dexelopment in psychophysics, in 
laboratory methods, and in psychometrics 

The attempt to apply this type of technology to social psycliol 
ogy was much too literal and failed to consider the appropriateness 
of the technique to the problem under in\estigation Hence the 
earlier efforts to test Freudian concepts were fruitless In industrial 
psychology, precision measures of isolated motor performance were 
inadequate to cope with problems of fatigue and motivation The 
Item reliability technique of the psychometrician was no answer 
in itself to the need for measures of cognitive and motivational 
structure in dealing with social attitudes 

The real problem is not that techniques cannot be adapted to 
a variety of problems but that they tend to carry with them the 
type of thinking and even the concepts of the area in which they 
were developed Thus, the experimental technique when first 
applied to social psychology attempted manipulation of the amount 
of social stimulation— i e , the sheer physical dimension, as in the 
alone and together experiments The creition and manipulation 
of the specific social influences came is a later development Thus, 
when old techniques are used in the social field they have to be 
adapted to the conceptual framework in which they are applied 
Otherwise we shall find ourselves testing things other than the 
theories in which we are interested Moreover the special problems 
of our field call for new approaclie^^ ind new techniques The tradi 
tional measurement procedures involved assumptions not necessarily 
met by social data The development ot new scaling methods and 
of nonparametnc statistics ire hopeful signs of progress m this 
respect 

Finally, the social researcher sliouhl consider his research de 
sign from the point of view of testing the significant theories in his 
own field rather than from the frame of reference of what he would 
be doing if he were determining a sensory threshold It is our con 
\iction that methodologies need to be wrmcn for the field of social 
psychology itself 

Most of the contributors to this volume are social psychologists 
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This means that the problems they discuss tend to be taken from 
the field o£ social psychology. It k our belief, however, that there 
are many areas in the social sciences to which the approaches snd 
methods described in the following pages will have application. 
For areas in the social scienas which deal with relationships be- 
tween group indices t/ithout reference to intervening variables, 
these methods may need the same sort of adaptation to meet the 
criterion for appropriateness as was demanded in the field of social 
psychology when it took over techniques from individual psychology. 

A cooperative undertaking of this sort requires not only the 
assistance of the contributors of the chapters which follotv but the 
support of their colleagues. We are indebted to Donald Marquis, 
who participated in the planning of the project and who bears 
much of the responsibility for the circumstances which made the 
book possible. Other participants in the project not formally repre- 
sented in the following chapters were Eugene Jacobson, Lowell 
Kelly, Charles Meunec, Ian Ross, and Guy Swanson. In most books 
there is generally one person who carries the brunt of editorial 
work, and it has been our good fortune to have had Mrs. Emily 
Willerman for this role, which she has carried out with unusual 
devotion and competence. 

University of Michigan 
June 21, 1953 


L.F. 
D. K. 
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INTRODUCTION 


The Interdependence of 
Social-Psychological Theory 
and Methods: A Brief Overview 


Theodore M Newcomb 


It IS a truism that no research results are any better than 
the methods by which they are obtained Behind the platitude, 
however, lie many complexities Between the initial sensing of a 
problem axid. the Bnal application of research results to that prob 
lem, there lies many a dioice, as the reader of this volume will 
discover At each dividing of the path, moreover, there are diverse 
criteria for deciding what is “better” Just as Moliere’s M Jourdain 
was astonished on discovering that he had been speaking prose all 
his life, so not a few experienced researchers in social psychology will 
be amazed, on completing the baker's dozen of chapters that follow, 
to learn how many decisions they have been making these many 
years— With or without knowing it It is one of the objectives of this 
volume to create a more general awareness of the existence of the 
choice points and of the criteria by which decisions may be made 
No article of faith in the saentist’s credo is more elementary 
than his empiriast conviction that, if he learns how to ask the 
proper questions of “nature,” he can formulate the principles accord 
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ins to which “nature behaves If our questions are not properly 
put-i e , if our observations are not suitably made-m the first place, 
no amount of interpretative ingenuity at a later stage will enable 
us to reach our research objectives Such methodological problems 
as devising interview scliedules, selecting a sample of persons or of 
•wniten words, manipulating a variable in the laboratory, or con 
siructing an objective test are all problems of ensuring that the 
questions which we put to ‘ nature will be maKimally suitable to 
elicit the answers required by our objectives Problems of scaling, 
categorizing, discovering covariation, and testing the significance 
of differences are not merely matters of * translating data already 
obtained, they are basic in the seme of determining— whether we 
know It or not— the kinds of questions which we are putting What 
ever truth or falsity inheres in our research findings is quite as 
much a function of the questions we have elected to ask through 
our selection of methods as of the logic we have applied to the data 
elicited by our questions 

The kinds of research problems described in the following 
chapters have been attacked within a very limited time space setting 
The methodological weapons which have been devised have, like 
military weapons at a given time and place in history, been con 
ditioned by their setting Some aspects of this setting impinge alike 
upon every variety of research into human behavior, some have had 
a special impact upon social research, and still others have influenced 
in specific ways the somewhat dimly demarcated field of social 
psychological research This brief introductory chapter attempts to 
point to some o£ the comemporary methodological problems ior 
social research in general and for social psychology in particular, 
and to note the position of the social psychologist in the contra 
ternity of social researchers, as he borrows from and lends to his 
fellow members in the common enterprise 

SOME COMMON PROBLEMS IN 

SOCIAL RESEARCH 

Social scientists face certain human problems which the natural 
scientist IS spared As we shall note particularly in Part I these 
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problems begin with getting access to persons as sources of data It 
lias too often gone unrecognized that the problem is not just one 
of avoiding refusals, whether by doorstep respondents, by student 
subjects, or by representatives of organizations Such motivational 
factors on the part of the interviewees as those noted by Charles F 
Cannell and Robert L Kahn in Chapter 8 must be taken into 
account not only by interviewers but also by analysts in interpreting 
responses In a very literal sense, moreover, the conditions under 
which clients and respondents first agree to participate in an inves 
tigation determine the nature of the eventual findings Both respond 
ent and investigator, whether they know it or not, are taking roles 
The respondent’s initial structuring of this role relationship will 
influence, in conscious and in unconscious ways, both the fullness 
and the content of his later responses— just as initial orientations 
toward any object influence later behavior in relation to it As 
Ronald Lippitt and Rensis Likert note m Chapter 13, the process 
of research planning must include these facts of social life The 
investigator has made a research decision, whether he knows it or 
not, when he fint approaches a client, a subject, or a respondent, 
and he sets the stage for later decisions as he continues or modil es 
the initial role relationship^ 

One aspect of the role relationship between investigator and 
subject IS an ethical one In Chapters 3 and 4 , some of the uses of 
temporary dissembling are noted, together with the responsibilities 
imposed upon the investigator who uses them— responsibilities to 
colleagues, who may later have to pay a heavy price for his laxity, 
and to the ‘ consumers of his research findings, as well as to the 
subjects or respondents most directly involved The last of these 
obligations is probably most easily met, as Chapter 4 suggests, most 
subjects accept without resentment the fact of having been duped, 
once they understand the necessity for it Nevertheless, frequent use 
of what IS regarded as deceit may lead to community shared expec 
tations which undermine the necessary relationship of confidence 
between investigator and subject The attitudes of subjects recruited 
from such a community may be such as to influence their responses 
in ways quite unsuspected by the investigator and thus, perhaps 
invalidate his findings or his interpretations 

On a fundamental level, every soaal researcher, whether or not 
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he resorts to methods of dissembling stands m an ethical relationship 
to the community in which he works As Doctor ATM ^Vilson, of 
the Tavistock Institute, has said. It isn t so much that honesty is 
the best policy its the only possible policy The honest investi 
gator, who knows (or should know) more than the client about the 
consequences for the client of participation in the research, must 
not only not take advantage of his wider knowledge but must actu 
ally seek to turn it to the client s advantage The temporary duping 
of laboratory subjects does not necessarily violate tins concept of 
honesty whereas sheer thoughtlessness on the part of an investigator 
who would never think of lying to a client may violate it funda 

fnpntnllv ' 


Subjects and client, as ,vell as mvest.gators, have personal 
™lues which are apt to become involved in the research process 
explottable ts, to quote Dr W.Ison 

m L mtX t "s *”'• "" ' "P °P“ y™"- abdomen 

hort s based ‘“"u ' fundamental ethical pr.ne.ple. tn 
a* evelnt j ‘hut tn the long run the 

reirtTor h. ‘"''“‘■eutors objeettves ts dependent upon hi, 

appTcab” to the'tmnm ““ ts'^jus. as 

upon the latter ts much mom ihrea ‘“P”" 

which can protably L't.o^S^emmuall'y 'th*' °h 

inevitable in the eariv stn t though it seems almost 

ger IS bestUhtd 

pomorphize about humans /n ^ temptation to anthro 

pily outgrown fay most of us-it extreme form-now, hap 

by sheer intuition or empathy In 'u “iVe" 

form-and perhaps its most dangerous fnr contemporary 

and indeed inevitable-it resnifc seems natural 

cepts and variables selected bv r "®t >n observations but in con 

- -ell as the object investigator 
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morphic manner The tendenllT I ■" anthropo 
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be problems which set 
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our research mechanisms in action. But phenotypic phenomena are 
not necessarily the most significant ones to observe, even as de- 
pendent variables; and as intervening or independent variables they 
have, it seems to this writer, little better than chance likelihood of 
being the most significant ones. In clinical psychology, for example, 
a Rorschach W may be a more significant variable than the more 
humanly phrased “social expansiveness/' Just so, in social science 
we probably have more to gain by identifying genotypic X’s (with 
or without human-sounding labels) than by seeking to refine our 
measures of readily observable, human phenotypes. Helen Peak, in 
Chapter 6, has examined some of the properties of significant vari- 
ables in terms of “functional unity." 

Another problem of which social scientists of nearly every stripe 
are becoming increasingly aware has to do with the decisions they 
make when they employ a given process of measurement. In Chap- 
ter 5, Leslie Kish points to some of the consequences of using one 
device rather than another at various stages of sampling procedures. 
Keith Smith, in Chapter 12, points out some of the assumptions 
involved in the use of what may be our favorite statistical pro- 
cedures and suggests alternatives which many of us will find more 
appropriate, once we are aware of the nature of the statistical deci- 
sions we have been making. 

Clyde H. Coombs, in Chapter II, goes to the very roots of the 
question “What is the nature of measurement itself?” Since we are 
necessarily doing something when we transform "real events” into 
numbers— whether at the stage of making observations or at the 
stage of analysis— it behooves us to know what we are doing. The 
requirements of this transformation process, together with certain 
properties of the events which the social scientist studies, confront 
him with a special dilemma. This chapter is characteristic of the 
tone of the entire book: instead of presenting recipes to be followed, 
it seeks to understand the logic of a type of problem which social 
scientists frequently meet. Since for the social scientist every investi- 
gation situation includes a large component of uniqueness, and since 
(as Dor\vin Cartwight notes in Chapter 10, for example) the deci- 
sions made at every step of the investigation process are dependent 
upon decisions made at other steps, the investigator himself must 
construct his own blueprint. 
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SPECIAL PROBLEMS IN SOCIAL PSYCHOLOGICAL 
RESEARCH 


In addition to the hazards which beset all hinds of social 
research, social psychology is subject to some ‘ special disabilities 
sut genens Many o£ these stem from its interdisciplinary parentage 
and Its still ambiguous boundaries From the point of view of range 
and richness social psychology has gained much by its generous 
borrowings from individual psychology, sociology, cultural anthro 
pology, and psychiatry But the cost has been heavy. We draw upon 
a wide and only loosely integrated variety of concepts Our sources 
of data are as diverse as the analytic couch, the laboratory, the 
playground, the factory, the community, and the random sample 
of adults in a total society It is likely, indeed, that many social 
psychologists are not even aware that tliey have at their disposal 
so broad a range of materials as those noted by Robert C Angell 
and Ronald Freedman in Chapter 7 It is not surprising, therefore, 
that we have recourse to a wide range of methods m making obser 
vations, in isolating, measuring, and controlling our variables, and 
in analyzing our data Under such conditions the difficulties of 
discriminating among settings and methods of apparently equal 
relevance and serviceability increase exponentially with the range 
of alternatives Social psychological research seems, at midcentury, 
to be peculiarly subject to these difficulties 

There is, of course, nothing intrinsically undesirable in having 
recourse to a wide range of settings and methods, diversity, like 
adversity, has many uses Several o[ the following chapters note 
possible ways-or even necessary ones-ol taking advantage of this 
situation Thus it is pointed out m one or more of the chapters in 
l-art I that field studies have called attention to an increasingly wide 
range of variables which are manipulable or controllable in the 
in ^ Studies and surveys can supplement each other 

nlan* that certain kinds of problems are furthered by 

LnuZ finally. 

SaZd necessitate such 

ChTmer In somewhat similar vein Leon Festinger, in 

lahomo™. “ n' "aruficiahty in the 

laboratory as contrasted wtth the malness of other s.tuat.ons 
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It may happen, of course, that a specific investigator will spe- 
cialize in a given method sheerly bet^use of the conveniences or 
the necessities of division of labor. But this is not the same thing 
as specialization of method for a given problem. Hunches emerging 
from one sort of methodological study will often need to be tested 
by other methods, and results confidently confirmed by one method 
may turn out to be so loaded with the situational factors necessarily 
associated with the use of that method as not to be confirmable at 
all when other method-situation complexes are used. The more 
"basic” the problem, in general, the more essential it is that it 
be investigated programmatically by a designedly wide range of 
methods. 

Another, and doubtless closely related, handicap under which 
social psychology currently suffers is that of too rapid growth. If its 
development has not been exactly hypertrophic, the tendency has 
certainly been to incorporate more than has been digested. John 
R. P. French, Jr., obser%'es in Chapter S, for example, the relatively 
low levels of abstraction at which social-psychological propositions 
are of necessity frequently formulated. And Daniel Katz notes, in 
Chapter 2, the relative rarity with which social-psychological inves- 
tigations have been replicated. Tcian unknown extent both of these 
conditions are attributable to considerations of methodology. When 
we shall have achieved more satisfactory and more "standardized” 
methods of investigation, with clearer criteria concerning the range 
of appropriateness of various methods and tools, we shall, of course, 
be jjD a much better positicm to maXe gemtine})/ eo.'iiparabie studies, 
which are a necessary precondition to high-level abstractions. And, 
similarly, a more satisfactory armamentarium of tools and methods 
will facilitate the replication of significant studies. 

One’s assessment of the consequences of these and other present 
shortcomings of social psychology svill depend upon the nature of 
one’s hopes and expectations and upon one’s definition of the field. 
If one regards social psychology as an applied field, one will be 
dissatisfied with the present stale of affairs primarily on grounds 
that the applicability of a given principle to a gi\cn situation is 
highly uncertain. But if one looks at social psychology as itself 
constituting a body of thcoiy, the nature of one's concern will be 
quite different. Furthermore, the sources to which one looks for the 
improsement of meihotls uiJl \-ary considerably, according as one’s 
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definition of the field stresses its applied or its theoretical aspects 
This brings us to the reciprocal of our initial truism No method 
IS any better than the theory by which it is tested This point, 
touched upon in several chapters, is most explicitly made by Helen 
Peak, in Chapter 6, and Roger W Heyns and Alvin F Zander m 
Chapter 9 To the extent that our objectises center about applicn 
bility, we shall be content with empirical tests of their adequacy 
This way, however, does not lead in the direction of generalizability 
and high levels of abstraction Particularly m a field charactenzed 
by diverse kinds of settings and multiple divergent situational 
determinants this way leads, at best, to a cookbook like compendium 
of directions Even from the point of view of applicability it is a 
discouraging way, since the possible concatenations of situational 
variables are almost infinite 


Conversely, to the extent that we are moving toward objectives 
of high level generalizability, we shall look to theoretical tests of our 
methods as the crucial ones— le, crucial in ways such as those sug 
gested by Helen Peak But since this book is devoted to social 
psychological methods the question is thus raised concerning the 
existence or even the possibility of a genuinely social psychological 
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SOCIAL PSYCHOLOGICAL AS RELATED TO 
OTHER RESEARCH METHODS 

Suppose It be agreed that there exists an area of theoretical 
importance which does not he entirely within the confines of either 
sociology or psychology Let it be accorded a quasi independent 
status and labeled, for the time being, ' social psychology ’ And 
suppose It be also agreed that there exists a body of methods char 
acteristically used by social psychologists, not all of which are used 
by either sociologists or psychologists Does it follow that there must 
also exist a quasi independent body of social psychological methods’ 
It might follow from the demands of an exaggerated professional 
pride but not from those of logic Such distinguishability as social 
psychology has from its neighboring disciplines inheres in whatever 
distinctiveness characterizes the problems to which it addresses itself, 
the methods by which it attacks them might or might not be distmc 
tive In actual practice, there is little distinctiveness of method 

It IS evident, even from a quick scanning of the Table of Con 
tents, that the methodological problems treated in this book are by 
no means the exclusive or necessarily the primary concern of social 
psychologists If there are basic methodological problems m the 
planning of laboratory experiments, for example, social psychologists 
have not been the first nor will they be the last to be faced by them 
Social psychologists, being relative latecomers on the scene of 
research in human behavior., have been pioneers in few of the 
areas here represented, and in some instances their contributions 
ha^e been relatively minor ones Strictly speaking, there are prob 
ably no social psychological methods as such The opening chapter, 
by Angus Campbell and George Katona, for example, emphasizes 
the interdisciplinary nature of survey methods 

Nevertheless, the total contnbution of social psychologists to 
social research methodology has been considerable Precisely because 
their methodological problems have not been entirely distinctive, 
they have been in a position to lend What they have had to lend 
has not been of their sole creation They have had to borrow, but 
they have also had to adapt what they have borrowed, and in adapt 
ing they have invented It is their marginal status which has made 
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this both necessary and possible H'lving one foot in pioncei tern 
tory, they felt the necessity to in\eni having the other foot in tern 
tory at least partly explored, they had some awareness of methods 
available to be borrowed and adapted 

Nowhere is this better illustrated than m the still unfinished 
problem of quantifying qualitative data Sociologists and psycholo 
gists had long since developed their indices of discrete, objectively 
countable events and— borrowing from biologists— their statistical 
devices for analjzing their data But many of the data crucial for 
the social psychologists purposes were not discrete and countable 
Being unwilling to give up the advantages of quantitative control, 
and at the same time aware of the unsuitability of standard ' 
parametric statistics the social psychologist was forced to reconsider 
the whole research process This meant keeping m mind from first 
steps to last the interdependence of data gathering methods and 
the statistical methods by which they were to be analyzed Aware 
ness of such problems has resulted not only in the complete recasting 
of the methodology of attitude research but also in the development 
of new theories of scaling and even of measurement itself 

The margmality of the social psychologist s concerns may force 
him to become methodologically mvem.ve but tt is no guarantee 
a! ‘'i* “"’'■'t'ons There must be selectivity as ivell 

^ nventiveness, and selectivity presupposes a been sense of the 
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it has not been because of any jurisdictional label which its 
practitioners have worn. Rather, sensitivity to characteristically 
social-psychological problems has mothered invention because it 
presupposes the kind of cross-disciplinary familiarity which fosters 
discrimination among different sorts of data and among different 
sorts of methods. This, in turn, enhances the sense of appropriate- 
ness of methods to data and of both to basic theoretical problems. 


SIGNPOSTS TO METHODOLOGICAL 
DEVELOPMENT 

If it is true that research results are no better than the methods 
by which they are obtained, it is also true that the maturity of a 
discipline can be gauged by the spread of methodological sophistica- 
tion among its practitioners. By this test, social-psychological re- 
search is perhaps not very far developed, but its rate of development 
seems nonetheless encouraging. The publication of three works 
within as many years— the present one, the 1951 volumes entitled 
Research Methods in Social Relations (1), and a forthcoming Hand- 
hook in Social Psychology (3), each devoted in whole or in part to 
methods of social-psychological research— is both a sign of a felt need 
for such sophistication and a probable cause of its spread. One of 
the necessary conditions for the growth of science is the replicability 
of its findings; this depends upon standardization and codification 
of its methods, which, in turn, depend upon communication through 
books such as this one. 

Viewed in developmental perspective, the present position of 
social-psychological research thus seems a reasonably promising 
one. Among other factors rvhich will determine its position in years 
to come, one is of particular importance: methodological rcscarcli. 
At more than one point in this book the contributor has been forced 
to rely upon lore— f.c., common sense and the cumulative "wisdom” 
of experienced researchers. This, too, should be communicated, but 
it is no substitute for controlled research procedures. The "wisdom,” 
moreover, may turn out to be very unwise indeed. There has been 
considerable difference of opinion, for example, as to the advantages 
sometimes claimed for tlie facc-to-facc interview. But the necessity 
for basing our procedures upon mere opinions begins to disappear 
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when It is shown that certain specific differences, as predicted by a 
theory that barriers to communication vary with role relationships, 
distinguish interview responses and responses to printed question- 
naires by the same persons (2). 

The authors of this volume have been able to draw upon a 
limited pool of methodological research findings, to which they have 
themselves contributed. As social psychologists apply these research 
procedures to their own methodological problems, this pool w'ill 
continue to expand. 
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PART I 


Kesearch Settings 


Empincal research in social psychology, sociology, and re 
lated areas proceeds in a vanety of settings and contexts The choice 
of setting for any research project is generally guided by the nature 
of the questions being asked and the degree of control desired 

The variety of settings with which we must, then, concern 
ourselves may be ranged along a continuum from broad to narrow' 
We have chosen, for the sake of con\enience, to di\ide this range 
arbitrarily into four sections The boundaries between any two 
adjacent sections are ne\ er perfectly clear and sharp, but the di\ ision 
seems to have some \alidity in that different considerations and 
different techniques become important as we move from one to the 
other 

The broadest setting is, of course, one in which a large and 
perhaps spread out population is designated for investigation Here 
survey methods are usually employed Intensive study of field situa 
tions usually requires a narrower setting, such as a certain com 
munity or organization or industrial plant When experiments are 
to be done in real life settings, it is usually necessary to narrow the 



CHAPTER ONE 


The Sample Survey: 

A Technique for Social- 
Science Research 

A Angus Campbell and George Katona 


Many research problems require ihe s)slematic collection 
of data from populations or samples of population through the use 
of personal intervie^vs or other data gathering de\ices These studies 
are usually called surve)s, especially uhen they are concerned with 
large or uidely dispersed groups of people When they deal uith only 
a fraction of a total population (or unuerse) a fraction representa 
ti\e of the total, they are called sample surveys 

The basic survey procedure, as used in the social sciences, is 
made up of a combination of techniques uhich hate been developed 
in various research disciplines The procedures of interviewing, for 
example, are based largely on the experience of psychologists, 
antliropologists, and others who used the personal interview both 
as a research tool and as a means of diagnosis or therapy long before 
It was adapted for survey use Techniques of scaling and other 
methods of measurement have been borrowed from both sociology 
and psvchology Sampling methods have come in part from agricul 
tural economics Methods of content analysis have been drawn from 



16 Research Scltin 9 S 


a \ariety of fields, including political saeiitc Techniques of siaiisti 
cal analysis of mass data are common to all fields of quantitaiise 
research in the social sciences 

The survey instrument is not the specific method of any one 
social science discipline, and it is broadly applicable to problems in 
many fields It is this capacity for a\ide application and broad coter 
age which gives the survey technique its great usefulness in the be 
havioral sciences 


Surveys depend on direct contact with those persons, or a sam 
pie of those persons, whose characteristics, behaviors, or attitudes are 
relevant for a specific investigation Thus, the survey method differs 
from research carried out in libraries or archives by studying, re 


grouping, and analyzing records compiled for other purposes 

The survey technique is used only when the desired information 
cannot be obtained more easily and less expensively from other 
sources It would be very inefficient for example, to conduct a sam 
pie survey to determine the number of passenger cars in use in the 
United States Since American automobile ovv ners must register their 
cars every year, a study of the files of the state licensing bureaus can 
provide such information much more rapidly and reliably But no 
information is available from any records on the occupations, mien 

lions habits or other characteristics of automobile on ners Similarly, 

annual data are compiled, mostly from records kept for practical 
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information requires specialized techniques of determination, and 
few people would be 'able to report these facts with any degree of 
validity. Furthermore, it cannot always be assumed that people will 
be willing to give information which they may reasonably be ex- 
pected to have. Any information which might embarrass or incrim- 
inate the respondent is likely to be suppressed or distorted. Never- 
theless, the tolerance of respondents for inquiries into their persona] 
affairs is surprisingly high provided the interview is conducted 
skillfully and tactfully. This willingness of most respondents to give 
detailed information about themselves cannot be taken to mean, 
however, that the people who are asked questions in a survey will 
give answers with the impersonality of an electric tabulator. It is 
often easier to specify the data which a survey hopes to assemble 
than it is to devise and carry out the interview through which the 
data can be successfully obtained. 

Surveys vary greatly in their scope, their design, and their con- 
tent. As in any other research, the specific characteristics of any 
survey will be determined by its basic objectives. The statement 
of the essential questions which the research is intended to investi- 
gate delineates in large part the universe to be studied, the size 
and nature of the sample, the type of interviewing to be used, the 
content of the questionnaire, the character of the coding, and the 
nature of the analysis. Specific survey methods vary according to 
specific survey objectives. 


TYPES OF UNIVERSES SURVEYED 

The most widely known sample survey conducted in this coun- 
try is the poll conducted by the Gallup organization. This journal- 
istic enterprise has been asking questions of the American public 
since 1936, and its reports are a familiar weekly feature of many 
newspapers. Usually the Gallup Poll is intended to represent the 
total adult population; in its pre-election straw votes it attempts to 
represent the population of eligible voters. 

Although the Gallup Poll has been the most highly publicized 
survey of the national population numerous other nation-wide sur- 
veys are conducted each year. Perhaps the most ambitious of these 
is the Current Population Survey, carried out by the Bureau of the 
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Although geographical!) defined populations of these kinds are 
perhaps the most common basis tor sample surveys, there are many 
other uniterses which may be studied People of a certain occupa 
tion, for example, may be the subject of stud) The Department 
of Agriculture conducts a number of surveys each year in which it 
IS interested only m farms (38) Often it may define us universe 
as cotton farmers or soybean farmers, perhaps also restricting the 
area covered, so that the universe may include onl) so narroivly 
defined a population as farmers in the state of Illinois operating 
farms of more than thirty acres with soybeans as a mam crop 

Housewives are a frequently surveyed population, tisuaJJ) in 
relation to such information as preferences for packaged foods, meth 
ods and extent of canning and preserving of foods, incidence and 
size of home gardens, and the like Some household information, 
particularly financial data, cannot be obtained satisfactorily fiom 
housewives, and in such cases the universe would be defined is lie uK 
of households. 

Other occupational classifications have served as the basis for 
occasional surveys in which the objectives required information from 
such specialized groups During World War Jl, for example, there 
was a large scale program of survey research in the United States 
Arm), intended to answer a wide variety of practical questions rang 
ing from soldiers' preferences for different articles of clothing to 
their opinions on the determination of points for their ultimate 
discharge (42) During the same period there were a number of 
studies of shipyard workers, steel workers, and war plant workers 
of various kinds, undertaken to help solve the pressing problems of 
housing, transportation, absenteeism, and morale which existed in 
production centers during that period (26) 

Many surveys require samples of populations which are distin 
guished by some common behavior or experience There have been 
surveys of veterans, of college graduates, of subscribers to certain 
magazines, of visitors to state parks, of people who ride trains, and 
of many other equally specialized kinds of people Such samples 
we selected because these people have especial significance in rela 
tion to the objectives of the study For example, a Survey Research 
Center study designed to assess the factors which influence the 
purchase of homes drew its sample from a universe of recent home 
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purchasers people T\ho had only shortly before gone through the 
experience of selecting a home (47) 

Surveys may also base their samples on purely demographic 
characteristics A study of Negro reaction to discriminatory practices 
obviously requires a sample draiin from a universe of Negroes 
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ly few individuals in the total population are relevant The study 
lolitical leadership, for example, would need to concentrate on 
e infrequent individuals who fulfill the definition of leader 
^earch into the characteristics of legislative action would have 
^ oncern itself primarily with people seiving in the role of legis 
r An investigation of the behavior of business corporations 
lid require information from those individuals responsible for 
' isions governing the actions of such corporations For such studies 
“OSS section surve) of the general public would be entirely map 
priate 

^ A Survey Research Center study of factors affecting industrial 
^bihty provides an example of a specialized survey for which only 
itively few individuals could serve as respondents (25, 46) A 
■'jor objective of this study was to assess the influences which impel 
! lustnal executives to locate their plants at one or another of the 
lous sues available to them The sample vv^as selected randomly 
ni a list of all the firms located m a certain geographical area 
-'e state of Michigan) In each of the two hundred firms compris 
'5 the sample, a major executive was interviewed regarding the 
'Vantages and disadvantages of the location of his plant Decisions 
• relocating plants or selecting sites for expansion are ordinarily 
'5de by a few high ranking individuals and they are the only 
lalified respondents for a study having these objectives 
I Although the most impressive uses of the survey technique 
iring the past ten years have been in the sampling of large hetero 
I neous populations it seems probable that there v\ill be incieising 
j ^plication of this research method to populations of a more re 
ricted character As social science develops conceptual schemes 
^ised on observations and leading to new observations it is likely 
i3t the survey method (as well as ail other available research 
•ethods) will be applied more sharply to the study of people of 
^^eciahzed characteristics subject to specifically defined circum 
';-3nces 


types of survey design 

^Vhenever survey data are to be gathered there must be a deci 
as to the specific pattern or design which the data collecting 
'dl follow Every scientist attempts to arrange the conditions of his 
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research so that the data which are forthcoming will bear most 
effectively on the hypotheses he is attempting to test or the informa 
non needs he is trying to fulfill This is as Hue of the survey re 
searcher as it is of any other 


The Unweighted Cross Section 

The most familiar and the simplest survey design is the single 
time unweighted cross section This is the method par excellence 
for the determination of the characteristics of a population at a 
specific point in time For example, the systematic selection of every 

nth card from the register of the undergradnates of College 

would provide the basis for a description of that body as to age 
sex, high school record, college entrance test scores, college grades 
or any of the other items of information which appear on the sam 
pled cards Mean scores and distributions could be obtained for 
each of these characteristics 
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some subgroup o[ the clcsignnlcd unnerse uhich Ins especial impor 
tance for the objcctiNCS of the survc) but is kno\\n to be a relatively 
small fraction of the total population Thus, in designing a national 
survey for the stud) of the uses made of public libraries, the Survey 
Research Center unbalanced its sample design to bring into the 
sample a larger number of recent users of libraries than vsoiild have 
been caught in an unv\cightul cross section (10) This v\as done 
b) doubling the sampling rate in city blocks having high lenials, 
on the assumption that people who pay high rents arc moie likel) to 
visit public libraries The resulting increase m the number of library 
users interviewed made it possible to describe the characteristics 
and habits of this important group with greater confidence than 
would have been possible otherwise 

Oversampling is csjiccially useful in surveys dealing with the 
distribution and use of income and savings (43) It is well known 
that wealth is distributed unequally in this country, some individuals 
having incomes and assets many times the national average It is also 
apparent that in a sample of a few thousand households the influence 
that a relatively small number of these divergent cases can exercise 
on the total sample is appreciable To reduce the sampling error 
of the data obtained from the top income receivers, and to make 
possible the analysis of these people as a separate group of the total 
population, It is customary in such surveys as the Survey of Con 
suraer Finances to increase the total number of high income people 
by oversampling the areas in which they are most likely to be found 
Whenever oversampling of this kind is done, it is necessary, of 
course, to weight these cases down to their proper contribution to the 
total sample when the data are analyzed 


Contrasting Samples 

It IS sometimes more efficient to draw samples from subgroups 
which contrast in the variable most important to the study than 
It is to sample the entire universe 

This design is well illustrated in a Survey Research Center 
study of the influence on public attitudes of the proximity of atomic 
energy installations (16) The purpose of this study was to find 
out whether the establishment of atomic energy reactors tended to 
produce insecurity and apprehension m the surrounding communi 



24 Research Settings 


ties To provide the most effective test of this question, samples were 
interviewed in several cities situated within twenty five miles of 
major atomic installations similar samples were interviewed in 
cities paired with the atomic energy cities in geographical, indus 
trial, racial, and other characteristics but situated at some distance 
from any major atomic center 

A variation of the same design has been used by R C Angell 
m a stud) o£ moral integration of cities (1) On the basis of an 
examination of such indices as homicide rates aserage contributions 
to Community Chest, and the like Angell was able to order a list 
oE American cities according to an index of integration He then 
chose trio cities from both the high and lose extremes of this scale 
and interviewed samples from the population of each The purpose 
of this study was to ascertain the degree to which the factors which 
had produced the ongtnal differential, on of the cities as "well ' 
cuipf et*'/ reflected in the way people in these 
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science the range of change produang factors which are the subject 
of study IS very broad indeed Sometimes these factors are manipu 
lated as experimental variables by the researchers, as in the study 
of the effect of special preschool training on the I Q of young chil 
dren Very commonly, however, the social scientist is interested 
m the effect of events over which he has no control— for example, 
a declaration of war, an act of Congress, a race not, the advent of 
television, or a population movement Although these events are 
beyond his control, he can often anticipate their occurrence and 
arrange his measurements accordingly 

A simple example of the before and after design is found in 
a Survey Research Center study done in 1946 at the time of the 
tests of atomic bombings on Bikini atoll (7) The major objective 
of this study vs as to measure the effect vshich this highly publicized 
event would have on American thinking regarding the atomic bomb, 
on public anxiety concerning its use, and on popular estimations of 
Its military significance A nation vside sample of adults vsas inter 
Mewed in June just prior to the Bikini tests These people vsere 
asked a senes of questions relating to various aspects of atomic 
energy and the atomic bomb In August, after the results of the tests 
had been widely publicized and discussed, a second sample similar 
to the first, was interviewed This second sample was asked the same 
questions which had been asked in June as uell as a number of 
special questions relating to perception of what the Bikini tests had 
shown A comparison of the results of the two surveys showed that, 
although there was some surprise that the Bikini bombs had not 
done greater damage, there vsas no essential change in public percep 
tions or attitudes regarding the aiomic bomb 

A similar design was used by the National Opinion Research 
Center in a stud) of the effectiveness of a campaign conducted in 
Cincinnati intended to educate the population of that cit) regarding 
the activities of the United Nations (40) A survey conducted before 
the campaign established baseline scores of public interest in tlie 
United Nations, information regarding its activities, and attitudes 
toward its accomplishments This study indicated the sections of the 
population v\hich shotted themselves to be most in need of enlight 
cnmeni'* A comparable sampling six months later, after tlie cam 
paign, demonstrated the extent to which people had been reached 
b) the campaign and had been influenced by it 
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The study of trends differs from the before and after design only 
in that more than two measurements are taken and the measure 
ments are spaced over a continuing period rather than on either 
side of a specific event One of the best known trend studies done 
b) sample suiveys is Cantril s study of changes in American attitudes 
in the period prior to Pearl Harbor toward entry into World War 
II (II) Through a series of surveys conducted during 1940 and 
1941, Cantril was able to trace the gradual rise of public sentiment 
or 11 to England, for resistance to Japanese aggression, and for a 
declaration of war on Germany 

In a different area of information, the British Survey of Sick 
ness reports each month on prevailing rates of sickness, incapacity 
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rcspondeni is asked to rcj>orl on lus medical expeiiences ol the 
prcMoiis tuo months, a su/ficicntly short length of time to minimize 
memory error Each month a new sample of 1000 adults is inter 
Meued. By combining the reports of each two successive months 
samples for the one month they both report, it is possible to create 
an effective sample of 8000 reports for each month 


Reinterviews 

There are some types of survey objectives which require succes 
sive interviews with the same indi\idua!s This may involve only one 
interview and one reinterview or it may mean a series of interviews 
extending over months or years 

The reintersiew design is used when it is necessary to follow 
the activities or attitudes of the same individuals through a specified 
time period This is illustrated by a study of buying intentions 
reported by Lansing and Withey m which a national sample of 
consumers was asked at the beginning of the year whether they 
planned to buy an automobile or other durable goods during the 
ensuing twelve months (23, 29) At the end of the year, these same 
people were interviewed again and were asked whether they had 
bought a car or otlier durables during the year past The objective 
of this study was to analyze the n iture of the planning and decision 
making preceding a purchase, and this could be done only by fol 
lowing through on the stated plans of specific consumers 

A Similar design was used by Campbell and Kahn in their study 
of the presidential vote in 1948 (8) One of the objectives of this 
study was to measure the final shifts from staled intentions to vote to 
the final decision on Election Day Although the aggregate changes 
from October to November could have been measured by successive 
unrelated samples in these two months, individual shifts could not 
have been analyzed without successive interviev\s of the same people 
before and after the election 

In some cases multiple inteiviews aie taken with the same 
respondents simpl) because the objectives of the survey are too 
extensive to be covered in a single interview In the Indianapolis 
study of fertility (49), for example, four separate interviews were 
necessary in each of the sample households to cover all the questions 
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which had to be asked In such a situation, there is no advantage 
in spacing the successive interviews and they will ordinarily be con 
ducted as close together as possible 

Samples which are interviewed repeatedly over an extended 
period of time are usually referred to as panels (30) The use of 
the panel design is perhaps best illustrated by the study of the presi 
demial vote conducted in Erie County, Ohio, in 1940 by Lazarsfeld, 
Berelson, and Gaudet (31) This elaborate project was intended to 
■follow the vagaries of the individual voter along the path to his 
vote and to discover the relative effect of various influential factors 
upon his final vote It used a sample ol six hundred people who 
were interviewed seven times in successive months May through 
November By this procedure n was possible to observe individual 
shifts in inclinations toward one or the other candidate during the 
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necessitates repeated additive reports if longitudinal data are to be 
gathered reliably 

An application of this feature of the panel technique is found 
in the monthly Current Population Survey This study uses the same 
respondents for six successive months and can thus estimate not 
only the total number of unemployed in the nation each month but 
also the total movement into and out of this group An increase ol 
100,000 in the total unemployed group from one month to the next 
might mean that all the unemployed in the first month remained 
out of work during the second month and were joined by an addi 
tional 100,000 previously employed people However, it might mean 
that 300,000 people unemployed in the first month went to work 
in the second month and their places in the unemployed group were 
taken by 400,000 people who had been at work during the first 
month This total movement of 700,000 people can be followed in 
the Current Population Survey because changes in the work status 
of the individual members of the sample are reported month aPer 
month 

Two factors act as deterrents to the use of the panel design 
The first is the virtually inevitable mortality which occurs m any 
population sample over even a brief period of time In a cross section 
of the national population, a loss of 25 percent or more is to be 
expected after an interval of one year A large part of this is due 
to people's moving from one place to another, some of it to refusals 
to be interviewed a second time, and the rest to the numerous cir 
cumsiances which beset the efforts of even the best field organiza 
tions A loss of this proportion does not necessarily result in a biased 
sample but it creates the possibility of serious bias 

The second serious problem associated with the use of panels is 
the possibility that the continued interviewing will so sensitize and 
change the respondents that they are no longer representative of the 
universe from which they were drawn In the study of the effect 
of the Bikini atom bomb tests, for example, it was thought inadvis 
Jible to reinterview the before test sample after the test for fear that 
the first interview would call the attention of these people to the test 
3nd stimulate them to follow the news of the event more closely than 
^ ey otherwise would ha\e One can easily imagine that respondents 
" o know they are going to be interviewed month after month on 
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questions of foreign affairs, for example, will consciously or other 
wise prepare themselves for the next interview 

The effect of these two factors on the data gathered from a 
panel can be measured by interviewing a control sample, inde 
pendent of the panel, usually at the end of the panel period If the 
data from this fresh sample differ from those of the last panel survey 
by more than would be expected from errors of sampling, there 
IS reason to believe that reinterviewing has introduced bias 


VARIETY or CONTENT STUDIED 

The versatility of the sample survey lies not only in the variety 
o! populations to which it may be applied or in the choice of designs 
which are available but also in the broad scope of data which may 
be gathered Any fact which respondents are able and willing to 
tell an interviewer may become the subject of a survey study 

Survejs may supply answers to the questions ' how many and 
how much Some such data arc available from public records- 
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to enrich our store o£ knowledge— that is, our understanding o£ what 
has happened and our ability to predict iihat tvill happen 

The content o£ survey questions may be classified in \arious 
^vays The following classification divides the total range of ques 
tions into four broad areas of content 

Personal Data 

Surveys often include questions regarding the sex, age occupa 
tion, education, religion, nationality, group membership, and many 
other personal social characteristics of the respondents Similarly, 
they may contain questions about the size of income, assets, debts, 
and other economic variables The purpose of these questions is not 
so much to determine the incidence of these characteristics in the 
population as it is to provide the basis for the analysis of the relation 
of sex, occupation, or income to other data obtained in surveys 
(9, 21. 24) 

Environmental Data 

In many surveys it is important to know certain facts regard 
mg the circumstances in which the respondents li\e These might 
include data about the character of the local neighborhood, the 
adequacy of the living quarters or the proximity of friends or 
relatives 

A study of library use, for example, isould need to determine 
the relative availability of library sen ices A study of family relations 
iNOuld probably require information on the location and degree of 
contact with parents and ‘ in laws ’ A survey of home accidents made 
by the School of Public Health of the University of Michigan in 
eluded a detailed investigation of the homes of the respondents— 
their lighting, floor plan, location of rugs and furniture, condition 
of stairs, and the like (36) Knowledge of such environmental 
facts IS often needed to explain the behavior which is the surve/s 
principal object of study 

Pehavtoral Data 

Many survey questions deal with the actions or behavior of 
respondents In the economic field, for example, spending and sav mg 
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(purchases of Government savings bonds, houses, automobiles, tele 
vision sets, etc) have been studied in surveys so as to determine 
the frequency of these activities in a given period and their relation 
to other activities, and to determine the characteristics of those who 
have engaged in them Behavior which is only partly economic is 
studied in surveys when questions are asked about geographical or 
occupational movements, vacation trips, or visits to physicians 
In the study of political behavior, survey questions about voting, 
uriting to one’s Congressman, soliciting party contributions, and 
the like are relevant 

The analysis of information getting behavior would require 
questions on newspaper reading, radio listening, television viewing, 
movie attendance, conversations, and other related activities The 
total range of behaviors which might interest a survey planner is 
obviously very wide 


Level of Information, Opinions, Altitudes, 

Motives, and Expectations 

This broad area of “psychological" data includes many of the 
most interestmg questions available to survey analysis It is also the 

siirvc^" ^ ^ lihely to be data available from non 

survey sources 

information is often necessary as 
Lsur^h,, ° " "“a ^ It dangerous to 

and It IS difncuirt^" “t' equally understood by everyone, 
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Questions regarding opinions and attitudes are illustrated by 
inquiries about what people think are the purposes, adiievements, 
or shortcomings of the United Nations, how they feel about the 
competence of the federal administration, or whether they prefer 
a sales tax to an income tax Attitudes are generalized viewpoints 
of approval or disapproval To determine the presence or absence of 
attitudes and the reasons for holding them— what public issues do 
what kind of people approve or oppose and why?— is frequently an 
important survey objective 

Survey analysts are not usually satisfied with obtaining infor 
mation on people’s attitudes regarding specific unrelated public 
issues, it is often more important to investigate patterns of attitudes 
and interrelations among different attitudes In this case we do not 
ask simply how many and what kind of people approve of the 
Korean campaign, or of the Atlantic Treaty, or of sending troops 
to Germany, we want, rather, to determine whether a distinct group 
of people emerge who approve of all these and other related policy 
measures m contrast to another group opposed to them In this 
way, general attitudes may be discerned and it may be determined 
which specific attitudes are and which are not influenced by these 
general attitudes 

The study of motives and expectations represents one of the 
most challenging areas of survey research The moti%e concept 
stands not only for the stated reasons for behavior- that is the 
answers to the questions why (e g , Why did you vote Repubh 
can? )-~butxDco-egener3]}y lor the forces jxppeJhngioactJon Tvpfc 
tations represent the time perspective of a person as it spreads for 
ward into the future— that is his opinions and attitudes about what 
Will happen as well as his intentions and plans 


FORMS OF ANALYSIS 

Surveys directed toward a joint study of personal and environ 
mental data, behavior, and attitudes are usually much more pro 
ductive than those intended to cover only one of these aspects 
Among the many possibilities of such inlegnted inquiries a few 
important ones will be discussed here 
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CoTTiparison of Di^erent Parts of the Sample 

Instead of studying the distribution of certain kinds of behavior 
or of certain attitudes among all people, the sample may be broken 
into several groups with the purpose of determining the differences 
in behavior or attitudes among these groups. For example, the 
analysis of people’s perceptions of and actions regarding inflation 
during 1950-1951 became more meaningful when low-income groups 
were compared with high-income groups. Similarly, studies have 
often been made of behavior, opinions, and attitudes of specific 
educational or age groups. Groups which are compared need not 
represent demographic classifications but may be real groups to 
which people belong, such as work grotips in factories or offices. 


Linking Behavior and Attitudes 

In some surveys the most critical analysis requires the comparh 
son of behavioral or altitudinal groups. Attitudes are thus not 
studied in the abstract but are linked to specific forms of behavior. 
In a factory, for instance, high-production employees may be sep- 
arated from low-production employees and their attitudes toward 
the company or their foreman compared. Or, instead of studying 
all people s satisfaction or dissatisfaction with public libraries, the 
survey may contrast the attitudes of those who use the libraries 
frequently with those who use them rarely or not at all. Contrasting 
atiiiudinal groups have been established by asking people whether 
I ey eel that they are financially better or worse off than a year ago. 
S)uch a division of American families has made it possible to study 
et er puri^asing of automobiles is or is not associated with the 
eeiing of being better off financially (22, 23, 28). 


The Study of Motivations 

taken^r^rl! direct question of why a certain action was 

motivatiLartorceT' Since"” 

vational forces mai norh ™'’*" 

not be recognized L them P'=°P'='* 

it is necessa?! to ^ f having contributed to a decision, 

^7 use a correlational approach. One example of 
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unearthing motivational forces though correlation studies has been 
mentioned earlier. When people were asked during World War II 
why they had bought war bonds, they gave many reasons (usually 
patriotic or investment reasons) but rarely mentioned that they had 
purchased bonds because they had been solicited to do so. A com- 
parison of people with similar incomes and occupations who had 
and who had not been solicited revealed, however, that many more 
of the former than of the latter had purcljased bonds. 

Similarly, inquiries about reasons for purchasing automobiles 
may shed light on many important factors which have at one time 
or the other contributed to the decision to buy a new car. When 
asked why they bought a car during the preceding year, very few 
people mention that their income had increased. However, com- 
parison of car buying of those who had an income increase with 
those who did not have such an increase reveals the existence of such 
a relationship (23). 

Another example is found in the analysis of factors contributing 
to investment decisions (building of new plants, purchasing new 
equipment) on the part of manufacturers. In a survey of top 
executives or owners of manufacturing plants, the question of why 
expansion was undertaken or expansion plans were entertained 
yielded a great variety of reasons. Further relevant factors were dis- 
cerned, however, by asking the manufacturers questions about their 
profit expectations and by studying the relation of their answers to 
the presence or absence of expansion plans. The joint use of cor- 
r^iation analysis and of the direct question of “Why?” proved fruit- 
fril in helping to explain why some manufacturers entertained 
expansion plans though they expected stable or decreasing piofits 


^taking Predictions 

Although associations or correlations between different variables 
o not necessarily show which is the cause and which the effect, 
Such studies may provide information useful for the study of causa- 
tion. If relationships are established, for example, between past 
income increases and purchases of durable goods, a prediction 
out future behavior in case of widespread and substantial income 
increases may be made even if the question about causation has 
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Comparison of Different Parts of the Sample 

Instead of studying the dmnbution of certain kinds of behavior 
or of certain attitudes among all people, the sample may be broken 
into several groups with the purpose of determining the differences 
m behavior or attitudes among these groups For example, the 
anal)sis of peoples perceptions of and actions regarding inflation 
during 1950 1951 became more meaningful when low income groups 
were compared with high income groups Similarly, studies have 
often been made of beha\ior opinions, and attitudes of specific 
educational or age groups Groups which are compared need not 
represent demographic classifications but may be real groups to 
winch people belong such as work groups in factories or offices 


Linking Behavior and Attitudes 

In some surveys the most critical analysis requires the compari 
son of behavioral or attitudinal groups Altitudes are thus not 
studied in the abstract but are linked to specific forms of behavior 
In a factory for instance high production employees may be sep 
arated from low production empIo)ees and their altitudes toward 
the company or their foreman compared Or, instead of studying 
all people s satisfaction or dissatisfaction with public libraries, the 
survey may contrast the attitudes of those who use the libraries 
frequently with those who use them rarely or not at all Contrasting 
attitudinal groups have been established by asking people whether 
they feel that they are financially belter or worse off than a year ago 
Such a division of Amencan families has made it possible to study 
whether purchasing of automobiles is or is not associated with the 
leelmg of being better off financially (22 23, 28) 


The Study of Motivations 
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not been entirely clanRcd Detailed inquiries about the circum 
stances in i\hich behaiior has taken place may result in statements 
which imply information on causation as People svho expect sub- 
stantial increases in income r%ill sa\e relatixely little 

Asking people about their plans and intentions provides another 
method of densing predictions of things to come It must be empha 


sized ho\\e\er that the translation of expressed plans into pre 
dictions is by no means a simple process If a suney determines 
for example that a proportion of all American families representing 
three million people plans to bu) a new car in a gi>en year, the 
prediction that three million new cars will be bought in that year 
IS not justified Plans are subject to change with circumstances and 
purchases may be made by people who at an earlier time did not 
plan to make them If however two consecutive surveys find that 
the number of prospective automobile buyers has increased from 
three to four million from one year to the next the statement that 
the automobile market is firmer in the second year than in the first 
may be justified Expressed plans represent attitudes prevailing at 
a given time and information about them increases our knowledge 
about the situation at that time The greater our knowledge the 
better is our ability to predict (23 28 29) 

In some cases survey data can be used very effectively to predict 
public reaction to events which are known to be forthcoming A 
survey among industrial workers at the end of 1942 for example 
showed that many of these people had no cash or other assets except 
government bonds (48) Since the survey also showed that many of 
these workers were making incomes on which they would hate to 
■" 1943 they would 

tax oWiS 8'"''™"'-^"^ bonds to meet their income 

purcha f and r d of >h.s k.nd regarding public 
\t a U bt a r "'“do during World 
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applicable to various fields and scientific disciplines It may be 
argued that certain surveys belong in the realm of psychology, others 
in sociology, still others in economics, or political science, or public 
health Although such classifications may be justified from a certain 
point of view, it must be emphasized that the survey method is 
essentially interdisciplinary or, to put it more accurately, that it 
contributes to the integration of several traditionally separate 
disciplines 

Among surveys which are primarily in the domain of social 
psychology or sociology are investigations of group belonging, of 
leader follower relations of family life, of occupational choice 
Studies of income, expenditures, and savings may be classified as 
economic surveys Studies of voting, of participation in political 
movements, and of the distribution of political attitudes may be 
thought to belong to political saence Surveys of the incidence of 
illness, uses of medical services, or the nature of public beliefs 
regarding health concern the field of public health 

Such classifications, however, are somewhat arbitrary All sur 
veys, even those in economics or public health, have somtthmg to 
do with peoples behavior If we ask about level of information, 
opinions, or altitudes, we are primarily interested in finding out 
how and why people behave as they do If we omit enumerative 
surveys of the census type (intend^, for example, to determine 
the number of employed workers or the number of farmers raising 
''heat m the United States), we may conclude that all surveys have 
some relation to psychology They are concerned fundamentally 
'vith peoples behavior— their social behavior, their economic be 
havior, their political behavior, their health behavior Although 
statistical reliability requires the grouping of individuals survey 
data always derive from individual reports Finally, psychology 
enters into the picture because of the method of surveys the con 

between respondent and interviewer represents an interpersonal 
relationship of the kind that has historically interested professional 
Ps)chologists 

It IS not permissible however, to classify survey research as a 
P^rt of psychology Survey research has no specific disciplinary 
anchor point It is being used by speaalists m all fields of behavioral 
science, being adapted in each case to the requirements of that 
field Survey data are broadening the empirical base of a variety of 
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fields They are also providing the raw materials for an increasing 
volume of cross disciplinary analysis, which, it may be hoped, will 
serve in time to help bring about a closer integration of the presently 
separate behavioral sciences 

Depending on the intent of the survey planner, surveys can be 
a tool of applied research or can have a function m what is consid 
ered basic research When data are collected which policy makers m 
business or government need for their immediate practical purposes. 
It IS customary to speak of applied research The survey technique 
has been widely used by businessmen in their research on markets, 
consumer preferences, buying habits, and the like Numerous large 
commercial research agencies are continually engaged in surveys of 
this kind for American business Various branches of the federal 


government have also found reason to conduct applied research of 
this kind, usually to assess public response to their programs and 
to find ways in which the prog;rams can be improved (4) 

The use of the survey method as a basic tool of the behavioral 
sciences has been discussed tn the preceding section A great variety 
of hypothesis testing has been done through surveys, for example, 
m studies of the relation of economic experiences and expectations 
to spending and saving behavior (22, 23), the relation of personal 
fmstration to aggression against minority groups (5), the relation 
of identification with political parties to attitudes regarding political 
issues (2) and the relation of distance to the transmission of rumor 


The distinction between basic and applied research, though 
clear^ut in some instances, often proves to be superficial IVhat is 
led basic research may have more significant practical uses than 

us!d to so,H technique is 

If reliabl ^ F ^ ynamics of inHation-as it is, m fact, being used 

Luerw,'" “a “bout the factors which 

and in exre consumers to slock up or to buy in advance 
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Similarly, the extensive surveys conducted among members of 
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the armed forces during World War II may be regarded, from one 
point of view, as applied research. Some of these surveys— for exam- 
ple, those concerned with the attitudes of and toward Negro 
soldiers— were intended to assist policy decisions regarding the 
integration of Negro and white troops (42). At the same time, these 
studies can correctly be regarded as basic research, since they serve 
to clarify the possibilities of changing attitudes through personal 
experience and manipulation of the environment. 


FLOIV CHART OF A SURVEY 

The sequence of the tasks involved in carrying out a survey, 
from the first stages of planning to the preparation of the final 
report, is presented in this section. 

f. General Objectives 

The problems which make a survey necessary and the general 
objectives of the survey are stated. This statement is usually ex- 
pressed in broad terms and defines only the general area and scope 
of the project. 

2. Specific Objectives 

Although the general objectives, usually few in number, are 
ormulated without regard to the requirements of the survey tech- 
these are considered when the general objectives are broken 
own into the usually numerous specific objectives. The specifica* 
tion of all data to be gathered and of the hypotheses to be tested 
y the survey is accomplished at this stage. 

■5* Sample 

Two decisions must be made regarding the survey sample: (1) 
r universe of the survey is to be (will it be all American 

atnilies, or all employees of a factory, or all the physicians living 
ttt a region of the country?) and (2) the size and design of the 
*3ntple W’hich is to be drawn. After these decisions are made, the 
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actual draising of the sample units and the preparation of deline- 
ated maps, block lists, and the like may proceed 


4 Questionnaire 

The method b) which the sample is to be contacted (by personal 
mtervieiv, telephone, or mail) is determined, and a questionnaire is 
prepared The questionnaire is not simply a translation of the 
specific objectues into language understandable to the respondents, 
It IS built carefull), with regard to the type of questions to be asked, 
the degree of probing the sequence of the questions, and the estab 
lishment of rapport The draft of the questionnaire is pretested m 
the field before its actual use 


5 Field Work 

When personal interviews are to be conducted, interviewers 
must be trained both in general interviewing procedures and in 
questions specific to a given survey Interviewers are supplied vsith 
an instruction manual which explains the objectives of the study 
and the meaning of each question Provision is made for careful 
supervision of the interviewing 


6 Content Analysts 

The data obtained m a suney may be so simple that the inter 
tinr , =”<• directly transcribed into tabula 

Bm m '"ds through which tabulations are made) 

attitude and require careful editing, and 
by prenarimtT!ld ” ntquire content analysis This is done 

the'^XnTI reef V subsuming all 
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sis plan, which, in case of the use of machine tabulating equipment, 
results in writing out "machine requests,” contains the machine 
runs which are needed to test the hypotheses enumerated m Step 2. 
This plan has been implicit in the surveyor’s thinking from the 
very beginning of the study. His anticipation of the tabular material 
necessary to answer the objectives of the survey underlay the prep 
aration of the questionnaire and the determination of the content 
analysis. 


8. Machine Tabulations 

The results of the coding process are used to prepare punched 
cards, and the tabulations foreseen in Step 7 are carried out. 

9. Analysis and Reporting 

The data are analyzed, their reliability is determined, and a 
report is written embodying the survey findings. Sometimes, in the 
case of administrative or applied studies, survey findings are used as 
the basis of conferences with policy-makers for the interpretation o 
the implications of the research data for action decisions. This type 
of reporting is sometimes called “feedback. 

This scheme of the sequence of survey work should not imply 
that the nine steps are independent of one another. Some of the 
steps listed in succession are usually carried out simultaneously-- 
for example, the code may be prepared before the field work, and 
content analysis may be carried out at virtually the same time as 
interviewing. The survey process is a highly interconnected chain of 
events. Decisions at each step must be congruent with what has gone 
before and must anticipate what will follow (32, 34). 


RELIABILITY AND VALIDITY 

The reliability of survey data can be measured in the same \vzy 
as the reliability of any other kind of research data, by retest. Both 
individual and aggregate scores can be analyzed in this waj. 

Unreliability in sur\'ey data results from a combination of dif- 
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ated maps, block lists, and the like may proceed 


4 Questionnaire 

The method by which the sample is to be contacted {by personal 
interview, telephone or mail) u determined, and a questionnaire is 
prepared The questionnaire is not simply a translation of the 
specific objectives into language understandable to the respondents, 
It IS built carefully, with regard to the type of questions to be ashed, 
the degree of probing the sequence of the questions, and the estab 
hshment of rapport The draft of the questionnaire is pretested in 
the field before its actual use 


5 Field Work 

When personal interviews are to be conducted, interviewers 
must be trained both in general mtersiewing procedures and m 
questions specific to a given survey Interviewers are supplied with 
an instruction manual which explains the objectives of the study 
and the meaning of each question Provision is made for careful 
supervision of the interviewing 


6 Content Analysis 

The data obtained in a survey may be so simple that the inter 
views received may be easily and directly transcribed into tabula 
tions (or into punched cards through which tabulations are made) 
But even surveys of the census type require careful editing, and 
attitude and opinion surveys require content analysis This is done 
by preparing a code a numbered list of major items subsuming all 
the responses received to each question Coders must be trained, and 
coding must be supervised and its reliability established 


7 Analysis Plan 

The questionnaire of a large scale survey may contain 50 100 
or m questions It would be very inefficient to tabulate the reb 
tionship between the responses received to each question The analy 
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ferent types of error Interviewing error arises from inconsistencies 
in the wa) m which the interview is conducted Reporting error 
results from \ agarics of mood or attitude on the part of the respond 
ent Sampling error is implicit whenever a sample is taken as rep 
resentative of a universe Errors in coding, tabulation, and analysis 
make their inevitable contribution to the total Anything which 
tends to create different results under theoretically identical condi 
tions may be said to be contributing to unreliability 

The reliability (or consistency) of the information given by an 
individual respondent can be assessed either through related ques 
lions in the same interview or through the same question in sue 


cessive interviews In the former case, for example, the reliability 
of the reporting of personal financial data can be estimated by 
balancing the income and expenditure data which are given in 
individual interviews In the same ways respondents’ reports of 
their age can be checked against their report of their educational 
and employment history Questions intended to measure degree of 
information can be ordered m a scale of difficulty and the extent 
of mronsistency of response within individual interviews noted The 
re la i ity of altitudinal responses can be assessed m a similar way, 
using scaling procedures of the type developed by Guttman (41) 
Such measures involve some ambiguity, since they reflect not only 
^ividual unreliability but also failure to achieve unidimensional 


o! mltdlT by the companson 

kmf flh ■" '"volves special prob 

.!m irn^ „ ° '“b °‘ber within a very short 

his earlier amw"' I 'b^t the respondent will remember 

appear <hem verbatim in order to 

error may be introdu '/h ’*T interval elapses, 

the data he is asked "for inability to remember 

■f the dim :hS:thl°res^rd™ 
for a specific year, a point in a chanpntstS 

connt'ry’of ong,n-am ie^r.erwi‘tr~’“'’‘ h 

sistcncy(mer 90 percent id^ntitvt * ’"®b degree of con 

It has also becn^shown, liowcvL”7harr 

made a year after the end nf tl. ’ ‘ '^'^Ports of annual income 

y the end of the reported year often depart sob- 



The Sample Survey 43 


stantially from the report given immediately following the reported 
year and that these deviations tend to be biased in the direction of 
the individual’s income change during the year folloA\ing the ong 


mal report (50) 

Memory errors of this kind are so pervasi\e that there is a 
strong tendency in survey studies to reduce to a minimum the period 
the respondent is asked to recall Income data are usually requested 
on a yearly basis, and preferably shortly after the end of the calendar 
year, when income tax obligations compel most people to total up 
their year s earnings The Survey of Sickness, as we have seen, uses 
a reporting period of two months Surveys of radio listening cus 
tomarily ask for a report of only the previous day Generally speak 
mg, the more ephemeral and less eventful the experience to be 
reported, the shorter the reporting period 

The reporting error of attitudmal responses is difficult to assess 
in successive surveys because of the problem of differentiating true 
change which may have taken place between surveys from simple 
inconsistency of report For example, the same question ( Consid 
ering the country as a whole, do you think we will have good times 
or bad times or what during the next 12 months^ ) was asked twice, 
at the beginning of 1948 and at the beginning of 1949 From a 
sample of 655 identical respondents, it was found that 41 percent 
gave the same answers and 18 percent gave radically different an 
swers (22) The distribution of the aggregate scores was quite similar 
at the two dates Nevertheless, some of ibe consistency (41 percent) 
may have been due to chance and some of the change (18 percent) 
may not have been true change There has not been sufficient re 
search reported to justify any generalizations as to whether certain 
types of attitudmal responses are more or less persistent than others 
or are reported with greater or less reliability by individual 


respondents 

It IS customarily found that the consistency of aserages or 
frequency distributions is greater than that of individual scores 
In \Viihey’s study, for example, the distributions of 1917 urban 
incomes were found to be quite similar m surveys conducted early 
in 1948 and early in 1949 (the same respondents were intcrMCW^ 
m both suraeys), although the individual scores which make up the 
dutnbutions were far from perfect!) correlated as Tabic I indi 


cates (50) 
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Comparison of Income Data Obtained in Two Successive Surveys 

Proportion in Indicated 
Bracket in First Survey 
tn in 

From Survey Proportion Adjacent Nonadjacent 

7Afoney Conducted m snSame Bracket tn Bracket tn 


1947 Money 
Income Before 
Taxes 

Under $1 000 
$1 000-51 999 

2 000- 2 999 

3 000- 3 999 

4 000- 4 999 

5 000- 7 499 

7 500 and over 


in tmame nraciteitn 

Early Bracket tn Second 
1949 Both Surveys Survey 


* Less than one half of 1 percent 

I948'and^onL”m t” mtervieised once in earlj 

their 1947 income information about 

fLS R esearch Center 

independent distributions from comparable but 

total sample of a '‘eadily demonstrated by splitting the 

Table 11 the renorted"^^^ randomly selected subsamples In 
in a national su^ey of atumde^ respondents interviewed 

to the distribution! obtameH / business is compared 

fourth respondent n7\ Th combining the reports of every 

characteristic of survey dam appears is not un 

viesvmg methods careful sampling and inter 

cessive surveys ca^be d! of distributions from sue 

such as income or e^tZnT'^. demographic data 

- well Illustrated data This 

reasons people gave for the C buying (13) The 

during World War II occurr^'^^*^”*^^*'^ ^ interest in selling bonds 
survey after another (Table III) *” similar proportions in one 
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TABLE 11 

Income Otsfribuflens of Systemaflcalfy Sefecfed 
Subsamples of a National Sample 



A 

B 

C 

D 

Total 

$ 0-$ 999 

11% 

8% 


9% 

3% 

1.009- 1,999 

11 

13 

12 

9 

11 

2,000- 2,999 

23 

18 

19 

21 

20 

3 OOO- 3,999 

19 

19 

20 

20 

20 

4.000- 4,999 

11 

14 

11 

13 

12 

5,000- 7,499 

12 

15 

16 

14 

14 

7.500- 9,999 

5 

4 

4 

3 

4 

10 000 and over 

4 

3 

4 

4 

4 

know 

1 

2 

1 

2 

2 

Not ascertamed 

3 

4 

3 

5 

4 


ioo% 

100% 

100% 

100% 

100% 

Number of 






interviews 

317 

SOB 

299 

SOS 

1,227 



TABLE III 




Reasons Attributed to Government for Wanting to Sell Bends 




Jan 

June 

Nov 

June 

Reasons 


1944 

1944 

1944 

194> 

To finance the war, to win the war. 




to help soldiers 


65% 

65% 

67% 

68% 

To prevent inflation 


14 

15 

15 

14 

To get people to save 


7 

8 

7 

10 

0 proMde postwar security 

2 

3 

2 

3 

'-'ther reasons 


12 

9 

9 

5 



100% 

100% 

100% 

100% 

Number of interviews 


1 441 

1 925 

2 148 

2 263 

I^espite the many opportunities for errors to 

enter the 

surv ey 

rioLcss, there is no 

doubt 

that when surveys are 

conducted with 


proper observation of the basic tenets of research, many types of 
can be collected with not only tolerable but reassuring 

reliability 
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The \aIidation of survey data often presents serious problems 
The customary procedure for establishing the validity of measure 
raents made in social researcli is through a comparison with an 
outside criterion Unfortunately, there is not aUsays an acceptable 
criterion available when survey data are gathered The survey is 
likely to have been done precisely because there were no relevant 
data at hand 


Validation on an individual basis can be achieved by direct 
comparison of the information given by individual respondents 
with records available from other sources In a study reported by 
Cahalan, examination of courthouse records, automobile rcgistra 
tions, and other official documents showed a high correspondence 
with the report of individual respondents, although some items 
showed greater discrepancy than others (3) Not all differences in 
such a comparison can be regarded as survey inaccuracies, because 
official records are often not entirely up to date or complete and 
may not yet show changes which are reported in the survey 
A similar kind of validation can be made when respondents of 
known characteristics are selected as the sample and they are sub 
sequently interviewed regarding the charactensiics m question 
Hyman has reported such a study m which a sample of people who 
had c^hed government bonds during World War II were inter 
viewed to determine the uses to which this money was put (20) 
percent of these people denied that they had cashed 
y on s This relatively high discrepancy reflects the problem 

people are asked to reveal information which is 
not entirely to their credit 

comnarin"^™°" method of demonstrating survey validity consists of 
the meced d^rributions with comparable distributions from 

nbu If f sune, reports dts 

sex f e eduf" """'“Sn.ph.c character.sl.es of tts sample-age, 

data can atthel”” comparisons tsith Census 

data mn. at the least, tllnstntte the absence of moss errors 

ug™maTrdr — ■" ““P-"® 

few aeereeatp ^ ^''fortunately, however, relatively 

n.th ontstde est.ma.es and "h 

complex adpstments for i.Serent. Tn” 
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e\er, it is possible, for example, to check the \alidity of the number 
of births reported by a sample of families for the preceding year 
by comparing this figure ^sith the national birth rate as compiled 
from official records of the Office of Vital Statistics Or the number 


of automobiles reported owned by a sample can be expanded to a 
national total and checked against automobile registration figures 
The most detailed comparisons of this kind are those of income 
data Selma Goldsmith has reported comparisons of aggregate an 
nual incomes, demed from several sample surveys conducted by 
different organizations, with the national income as estimated by 
the U S Department of Commerce primarily from available records 
such as payroll statistics (18) In these comparisons, which required 
very complex adjustment procedures, the discrepanaes from cri 
tenon figures were found to be relatively small, mostly within the 
range of the sampling error 

Although many examples could be given of reassunng compan 
sons of survey data with outside figures, it must be recognized that 
Jarge discrepancies are also occasional!) found, despite observance 
the most rigorous standards of survey methodology In some cases 
^t seems virtually impossible to overcome reporting error, which 
tnay result in very serious bias This is relatively uncommon, how 
C'er, and is usually associated with deliberate falsification (as in 
the report of currency on hand) or with memory errors (as in the 
report of amount of bank deposits held year before last) 

The validation of data on attitudes, expectations intentions, 
and tha cA -a Vvwd Iw ‘Lhn viiwe: 

rs no outside "true score which can be taken as a criterion The 


statement v\hich the respondent makes at the time of the interview 
ts m Itself a datum for analysis It may or may not correlate with 
other nonverbal behavior Its usefulness for predictive purposes is 
^oater if it does, but its intrinsic validity is not dependent on this 
'■alidity IS present of course if the respondent consciousl) or 
nerwise misrepresents his attitudes or intentions because of un 
hngness to cooperate with the interviewer Such misrepresenta 
t'On IS often ver) difficult to detect 

Despite the absence of outside criteria, there are man) internal 
, can be applied to attitudinal data to demonstrate that 

are more than expressions of idJe caprice on the part of the 
tespondenis One finds, for example, that approval of proposals for 
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the government to provide jobs for everyone is much higher among 
low income people than among high, that intentions to bu) durable 
goods are higher among people v ho expect increases in income than 
it is among those who expect decreases, that expressions of hostility 
tow'ard Negroes are mo’-e common among people in Southern dues 
than among people m Northern ciues, that people who disapproved 
of the TaftHartle) Act in 1948 were much more likely to vote 
Democratic than those who approved it. These findings are con 
sistent with our general expectations based on other informauon 
about our soaety and its functioning This is by no means as con- 
vmang a validation of survey data as an established outside cri- 


terion vvould provide In the absence of such criteria, however, 
analysis of the interna! logic of survey data can often present an 
impressively consistent picture. 

^\Tien survey data are used for purposes of prediction, addi- 
uonal questions of validity are encountered. A respondent’s state 
ment in Oaober of his intenuon to vote may be a v’ahd expression 
of his mclmatton at that time but an invalid indication of his aaual 
ote in Noimber. A comumtr's exprtssion in January' of his 
mention to buy a at during the ensuing tuehe months may be 

COnuLen Unfol-BEW 

fact Z Sere Carrying it out The 

pressed bv ^rrespondence between intentions ex- 

not mean of contT^ ccspondents and their subsequent actions does 

of .nie^n;:! rn”'; torr„e''Tor‘'r' 

lions to bus rliirrsM J example, the trend of inten- 

samplc) determined 'inTht Pcoponions of the total 

has prosed to be indiattise ot .1. ^ Consumer Finances 

Schsseigcr has shosm, also tha, ih!s subsequent purchase 

good not onlv for thp ' i ^ ‘^^respondence of trends holds 

for example (37) ^ people or skilled workers. 


limitations 

technique It^ cfearlf^ '“P' 

) escaren instrument of great seisatilir 
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applicable to a wide range of problems in the general area of social 
research It has evident limitations, however both m precision and 
adaptability 

It IS apparent that any data gathering project based on a sample 
IS subject to sampling error This means that all findings coming 
from such a study must be interpreted in the light of this error 
This limitation becomes particularly important when a total sam 
pie is divided into parts for purposes of analysis Every researcher 
who has dealt with data from sample surveys has experienced the 
frustration of running out of cases as he pursued his analysis into 
smaller and smaller cells This is especially true when the analysis 
calls for division of the sample into geographic regions Few samples 
are large enough to permit regional analysis on any but the broadest 
basis 


Survey research is also subject to all of the errors of measure 
roent implicit in any attempt to derive a score representing a 
person s attitudes, abilities, behaviors, or other traits Since surveys 
usually depend on the voluntary cooperation of respondents it is 
often not feasible to use the tedious psychophysical methods which 
*uight seem desirable for the greatest reduction of measurement 
error Compromises of this kind may not be necessary when the 
subjects of study can be coerced (college students or military per 
sonnel, for example), but they may well be when the respondents 


are randomly chosen adults in their own homes 

Despite the large number of questions that can be asked in a 
single survey, there are limits to the number of topics that can be 
covered Every data gathering instrument has an optimal length 
for the population to which it will be submitted Beyond that point, 
tnierest begins to lapse and cooperation to dimmish The survey 
researcher must take care that he does not overestimate the tolerance 
uf his respondents, and he commonly finds that this involves sacri 
icing questions to which he would like very much to have answers 
The limitation on length may be especially important m areas 
which are thought to require a long period of questioning before 
rapport can be established and resistance overcome This might 
^Pply, for example, m the study of personal mahdjustmcnt or 
discord, where clinical experience might lead one to expect 
a first micrvicvv of an hours length would not be sufficient to 
uncover the underlying problem This seems a reasonable assump- 
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the government to provide jobs for everyone is much higher among 
low income people than among high, that intentions to buy durable 
goods are higher among people who expect increases in income than 
It IS among those who expect decreases, that expressions of hostility 
toward Negroes are more common among people in Southern cities 
than among people m Northern cities, that people who disapproved 
of the Taft Hanley Act m 1948 were much more likely to vote 
Democratic than those who approved it These findings are con 
sistent with our general expectations based on other information 
about our society and its functioning This is by no means as con 
vincing a validation of survey data as an established outside cn 
tenon would provide In the absence of such criteria, however, 
analysis of the internal logic of survey data can often present an 
impressively consistent picture 

When survey data are used for purposes of prediction, addi 
tional questions of validity are encountered A respondent s slate 
ment m October of his intention to vote may be a valid expression 
of his inclination at that time but an invalid indication of his actual 


vote in November A consumers expression in January of his 
intention to buy a car during the ensuing twelve months may be 
a true representation of his intent but there are many unforeseen 
contingencies which may prevent him from carrying it out The 
fact that there is not exact correspondence betsveen intentions ex 
pressed by individual respondents and their subsequent actions does 
not mean of course that trend data derived from repeated surveys 
of intentions cannot be of value For example the trend of inten 
tions to buy durable goods (expressed as proportions of the total 
sample) determined m the annual Survey of Consumer Finances 
nas proved to be indicative of the trend of subsequent purchases 
c weiger as shown also that this correspondence of trends holds 
irrni ^ population at large but also for various sub 

for eLmp‘le''(37)'’“'‘‘“°‘’ 


limitations 


The 

technique 


foregoing pages have illustrated the scope of the survey 
is c early a research instrument of great versatility 
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applicable to a ivide range of problems in the general area of soaal 
research It has evident limitations, hot\e\er, both in precision and 
adaptability 

It IS apparent that any data gathering project based on a sample 
IS subject to sampling error This means that all findings coming 
from such a study must be interpreted in the light of this error 
This limitation becomes particularly important ^vhen a total sam 
pie is divided into parts for purposes of analysis Every researcher 
who has dealt with data from sample surveys has experienced the 
frustration of running out of cases as he pursued his analysis into 
smaller and smaller cells This is espeaally true when the analysis 
calls for division of the sample into geographic regions Few samples 
are large enough to permit regional analysis on any but the broadest 
basis 

Sur\ey research is also subject to all of the errors of measure 
roent implicit in any attempt to derne a score representing a 
persons attitudes, abilities, behaviors, or other traits Since sur\eys 
usually depend on the ^olunta^y cooperation of respondents, it is 
often not feasible to use the tedious psychophysical methods which 
ought seem desirable for the greatest reduction of measurement 
error Compromises of this kind may not be necessary when the 
subjects of study can be coerced (college students or military per 
sonnel, for example), but they may well be when the respondents 
we randomly chosen adults m their own homes 

Despite the large number of questions that can be asked in a 
stngle survey, there are limits to the number of topics that can be 
covered Every datagathenng instrument has an optimal length 
for the population to which it will be submitted Beyond that point, 
interest begins to lapse and cooperation to dimmish The survey 
researcher must take care that he docs not o\ erestimate the tolerance 
of his respondents, and he commonly finds that this involves sacri 
Rang questions to which he would like very much to have answers 

The limitation on length may be especially important m areas 
''Rich are thought to require a long period of questioning before 
^Ppori can be established and resistance overcome This miglii 
®Pply. for example, in the study of personal maladjustment or 
family discord, where clinical experience might lead one to exp^ 

a first interview of an hours length would not be sufiicieni to 
“neover the underlying problem Tins seems a reasonable assump- 
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tion, although the Kinsey studies o£ sexual behavior indicate that 
under some circumstances a short interview can bring out peisonal 
information which is ordinarily carefully concealed (27) 

A sample survey designed to represent a population dispersed 
o\er a wide geographical area is likely not to give an adequate 
representation to any population characteristic which is highly 
localized This means that the influence of specific local social 
groups, for example, cannot be assessed through the usual national 
suHrCy, since it is unlikely that more than a very few members of 
any such group would be caught in a cross section sample The 
study of local community factors requires a concentration of effort 
on the speciflc community rather than the broad dispeision which 
IS desirable when a v,idely scattered population is to be lepresented 
It IS impossible to analyze adequately the complex fabric of 
social organization through the survey method alone, because the 
process of sampling tends to lift the individual respondent out of 
his social context Other methods are better adapted to the study 
of the countless interconnections which give society its integration 
It IS apparent that the survey method is not well suited to studies 
of historical development Ordinarily, survey reporting refers to a 
specific point in time or to a relatively short time period Studies 
o origins and longterm developments require research methods of 
a more longitudinal character 

the obvious limitations of the survey procedure arise from 

of manno “ requires a considerable investment 

a c3e T »“rvoys of highly localized and 

r ml bo successfully carried out by 

gence IC ‘ bas the requisite skills and d.h 

rmearchers " “""'^oration, a small group of 

hon^t e .de„„rh considerable piopor 

surveys are conducted by ‘^^pfo? so “7 
sometimes several hundred ™ 'ochmcians. 

specialists in study design samolm^ "^bis may include 

interviewing, coding madii’ne . P' ng questionnaire construction. 

The technofogy of surveying has “"olysis 

sional training m this f.efd toatfrav^H “ prote 

It the many pitfalls which av,a , *e u ™' 

I me untrained are to be avoided 
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The ponderous character of many surveys imposes a further 
restraint on the researcher which he may find irksome Once the 
design of a survey is set the survey must be earned through accord 
mg to those specifications This means that it may take months 
before a specific hypothesis can be tested and each new variation 
of the study design intended to carry the theoretical development 
further will require additional months To the laboratory researcher 
accustomed to varying his experimental design every month or so 
ihis slow pace may seem an intolerable frustration 

The full contribution of survey research to the development 
of the behavioral sciences however can be achieved only through 
continuing programs of research extending over a period of years 
The value of such programs is not primarily in their ability to 
repeat the same observations at different points in time Much more 
important is the opportunity they provide for the application of 
an integrated framework of theory to diverse aspects of behavior and 
for progressive revision and improvement of the hypotheses tested 


CONCLUSION 

Although the origins of the sample survey go back to the nine 
teenih century, the advanced methodologies now available to survey 
researchers have been developed only within the last few years 
None of the specific studies referred to in this chapter v\as done 
before 1940, few of them could have been done before that time 
The experience of the past ten to fifteen years has demonstrated 
to social scientists generally something of the potentialities and 
imitations of the sample survey There can be little argument that 
tils development has given an important stimulus to the quanii 
tative study of social phenomena There is reason to believe that 
tic l>ossibiItiics of the survey technique are still not fully realized 
either m scope of applications or in precision of methods 
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CHAPTER TWO 


Field Studies 


Darnel Katz 


Field studies are similar to nation wide surveys in that they 
have opened up new possibilities for the development of social 
psychology and the social sciences On the one hand, they are 
brcaling down the narrow walls of the traditional experimental 
laboratory in the application of a research approach to complex 
problems of human relationships The effect is twofold ( 1 ) our 
scientilic knowledge is increasing as a result of the direct study of 
field situations and (2) the psychological laboratory is beginning to 
include m its experimentation social and group variables 
r„r , 1 ," ™ ‘htt potentialities of surveys and field studies 

nbne. n”™ >=^00 greater These disci 

but thevT °”^i. and significant social problems 

on data^rnU cither on uncontrolled observation or 
they have Hern rather than scientific purposes Thus 

Slfratfr^ criL Rustics or 

data already ga’themd 'Fwld "t^ research designs are imposed upon 

duction of comrols and ‘'‘f ‘"S™ 

tion Itself Th,« tr, J research objectives into the data collec 

and the ttnes of nhs"* ' **”'** problem under investigation 

be under the mm i*^^^i!**"* measures to be taken can now 

Xr .rownT™ A science which can 

gainer m own data according to ns 


non to -ivTii.nc - "-•*••••5 »o us own research interests, in addi 
availing itself of existing mcords. is at a tremendous advan 
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tage over a discipline which ships this important part of the 
scientific process When the practicality of the additional costs of 
controlled data collection in the social sciences is established, we 
may witness revolutionary developments m these fields 


THE RELATION OF FIELD STUDIES 
TO SURVEYS 

Although It IS not easy to draw a fine logical distinction between 
a survey and a study of a field situation, there are practical differences 
which call for somewhat different techniques and skills The differ 
ence is roughly between the greater scope of the survey and the 
greater depth of the field study More precisely, two essential dis 
Unctions can be made In the first place, the survey always attempts 
to be representative of some known universe and thus attempts, 
both in the number of cases included and in the manner of their 
selection, to be adequately and faithfully representative of a larger 
population This emphasis on sampling may or may not be found 
tn a field study, which is more concerned with a thorough account 
of the processes under investigation than with their typicality m a 
larger universe In a survev we alwa)s ask about the relative inci 
dence, or distribution, of socnl variables or personality character 
utics in the larger group with which wc are concerned The 
explanation of trends in population inciease, of economic booms 
and depressions, of the amount of uncmplo)ment, of social change 
generally, must be understood m the context of the country as a 
''hole, so sampled that the many subgroups arc properly represented 
“md the relative weighting of factors as the) contribute to the total 
outcome, is ascertained 

A second and more important difference is iliai in the field 
>n\cstigition vse attempt to stud) a single community or a single 
group in terms of Its socnl structure— I e , the mtcrrclaiions of tlic 
J^rij of the structure and of the social interaction taking place (2) 
he siirvc), to the extent that ii deals with such interrelations and 
•ntcnctinn, does so through a stud) of the final outcome The on 
going social and psjchological processes arc mferred in llic survey 
ihcir statistical end effects In the field stud) hovvever, attempts 
nnde to ofiscrvc an<| measme the ongoing processes more 
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directly Specifically, this means that the field study either attempts 
observations of social, interaction or investigates thoroughly the 
reciprocal perceptions and attitudes of people playing interde 
pendent roles Thus a field study will provide both a more detailed 
and a more natural picture of the social interrelations of the group 
than does the survey 


Studies of attitudes toward labor management problems fur 
nish an example of these two types of approach A national cross 
section survey would report the incidence of attitudes toward labor 
and toward management for the nation as a whole and would seek 
to get some account of the distribution of these attitudes among the 
su poups m the population From such a study we would know 

worked n 'he nation, of 

of middle against workers not in unions, of farmen, 

stu”v ctcern'd^'^r','"'’ ’"dustr.aUsts A field 

pTam and would "" f"' P™'’'™ "”Sht deal w.th a single 
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the union a second f influence and communication within 
company still a third ‘"'Shi he similar relations within the 
lures Systematic interwe ^ 'he relations between the two struc 
perceptions and attitude^'?® '™'' attempt to get at the reciprocal 
officials, and some observaiion°'^*^T’ rewards and higher 

occurring m the factorv h "“Ah' he planned of the interactions 

I<cward,®tew:r: anTL^remareV™^"'^ 
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compare w.th the' pop.dauon Is ^ :Ce"' Sroup, 

findings more wisely f22^ SernTi#ti l ^ could interpret the 
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processes at work in a certain social class, a drive toward power 
rather than toward economic security in the top economic brackets 
The confirmation of this hypothesis, however, is easier to achieve 
by working with a subgroup of the national population On the 
other hand, after definite motivational processes have been studied 
in detail in a subgroup, the conception of the variables may be so 
sharpened and their measurement so facilitated that it may now 
be possible to utilize measures of these variables in a national survey 


TYPES OF FIELD STUDIES 

There are many types of field studies, but the method has been 
most widely used by the anthropologist in his study of primitive 
societies Tlie sociologist, influenced by this type of natural observa 
bon, has made detailed studies of pans of Ins own society and has 
often been able to add some degree of measurement to the more 
interpretative anthropological appioach Finally, the social psychol 
ogist has emphasized the impoitance of quantification and venfica 
tion of observation even in studies conducicd outside the laboratory 
The most important dimension, tlicn on which field studies can 
'st-y IS the degree of measurement they represent, ranging from 
the extreme of the interpretative anthropological description of a 
primitive society to an investigation employing standardized quan 
tification of data collection in the form of observational scales for 
recording behavior and attitude sciles for the measurement of 
beliefs and feelings 

An illustration of an antlnopologio! field study which attempts 
^ functional analysis rather than i sheer descriptive account is to 
c found in B Malinowski s in\cstigations of the Trobriand Island 
ers (16) Malinowski lived among this Mehnesian people, obsened 
*cir activities at first hand and spent man) hours talking to a num 
’Cr of native informants He presents both a detailed description 
"'ud a sociological explanation of then economic activities, their 
social organization, their n7)tljs and ideologies, and their psjcholog 
character structure He reports, for example, the central role 
” reciprocal obligations in their economic, legal and ceremonial 
*uc 

The islanders, living near ibe shore, will have partners in the 
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inland villages so that there is constant interchange of fish and 
fresh vegetables carried out with due ceremonial and public display 
Similarly in the fishing village there is a mutual set of obligations 
with respect to the fishing canoe manned by a number of natives 
under the ownership of one man but with effective rights on the 
part of the crew with respect to time of fishing and a fair share 
in the catch and with duties toward the maintenance and care 
of the vessel Apparently the binding force of the social norms 
o t e community comes not from legal penalties or from sheer 
custom but rather from a sustained give and take 
with both parties to an arrangement benefiting thereby and from 
H,! of this relationship through ceremony and public 

present his partner with a lavish 
reeardM*}? i. ® 8®“** return in the future and is highly 

conclude th h's prowess and generosity Malinowski 

zTs r ;::reLc,s; 

It hcls'Thrfirfalft"'* plausible as Malinowski s inlerpertation is 
society The hvnoih° generalization even for Melanesian 

mSnlnce o - tr ™P°«ance of reciprocity for the 

in the Melanesian l •° *’e tested in systematic fashion 

-ay be functmmne tr 

some measurement of this * ’''''e-atic test it is necessary to have 
that Malmork!?f?,n ? T'T' >>e noted however, 

-any descriptive accounTol cultir« Th 

theoretical concents wh,^h of analysis utilizes 

interactions Hence the *^0 observable social 

testable hypotheses ” ^rpreiauon is readily converted into 

found in the Lynd s^^s^tuS^orM^H!!'!^ “othropological approach is 
hfe of an American commL ^''hich reconstructs the 

gallons of a small tt>^m basis of the intensive investi 


gallons of a small team of r,p\A ^ ^he intensive investi 

>n the community and particm^i^^i ^he field workers lived 

-.pioymg pariiclpau'; ZZ,o„ “ ' 

In this process informal intervtf>t * major methods 

tion they conducted a thorouf»h'*^*"^ occurred frequently In addi 
materials inclu examinano„ qj documentary 

school records MLSU "CENTRAL LIBRARY f records court files 
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not only tor the current period of the study but for earlier years 
as well Similarly, past and current files of newspapers 
suited, and histones o£ the state, county, and city were studied 
Considerable emphasis was placed upon understanding the com 
munity in terms o£ its history Dianes were examined tor the early 
years to supplement the other historical records Moreover, where 
no statistical materials were available, the field staff compiled data 
on wages, steadiness ot employment club membership, church 
attendance and membership, attendance at motion pictures etc 
This introduction ot quantitative materials was made more 
by carefully planned interviews with a sample ot working 
wives and a sample ot businessmen s wives Written questionnaire 
were also sent to more than 400 clubs and to three fourths ot the 

senior high school population , , j j 

From this mass ot data and source material, the Lynds derived 
their account ot the major activities ot the community, the t^iids i 
us development, the pattern ot conflicts in us life ey on 
differential rate of change in the performance of various basic 
activities, with the greatest changes in the economic pursui s 
getting a living The general pattern of change was from he 
busing class to the working class, with the working class ofte 
showing today the habits of the business class of a generation ag 
There are instances however when this process is reversed an 
practices of the lower income groups have been taken over v 
upper income groups In general, the currents o c lange sem 
erratic, with respect both to direction and to the differential rates 
for different aspects of life The difficulty of maintaining some sort 
of equilibrium under these conditions of stress emerges as a m ijor 
problem One solution which does appear, though it is not systcmati 
tally utihred is a sidling procedure in which an innovation fust 
appears as an optional allcrnate mode ot adjustment am t icn grat 

ually rephees tlie older mode ^ 

The lyiids have gone beyond the traditional anthropologim 
method in utilizing quantitative iccliniques to supplement tic 
qualu-iiue nntcrnls Their snicnicnis about the aciiuiics o ic 
rommumiy and the clnnctcrisiics of its people me o ten *^*^*'1; 
from statistical tables Their conclusions and their more iiiterprc 
• picture, howc\cr. arc drawn IicaMi) from a consu cration o 
•he cpnliiatue information and from tbcir own cvpcricnccs s 
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participant members of the community Thus not all of their 
conclusions can be readily confirmed by other investigators because 
of the difficulty of duplicating the exact procedures on tv Inch the 
conclusions are based In general however, in its attempt to docu 
ment its observations with facts and figures this study represents 
a real advance toward more objective scientific methods What 
emerges very clearly in this attempt is tlie distinction bettvttii 
findings or data and interpretations uliich are derived from the 
observations and data 


Middletown study m intenveavmg quan 
"Xerviews and questionnaires n.tl, 
=>"<1 general obsersat.ons and in 

frame of refer! “ h ■> "'°'‘e 
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personal relations, so Schanck found that the fundamentalistic 
beliefs and practices of the older religion were largely for public 
purposes In private, individuals maintained a much more libeial 
set of beliefs and attitudes Part of the conformity on public occa 
sions was a matter of pluralistic ignorance, the erroneous belief held 
by many that the rest of the community felt differently from the 
way the respondent himself felt Also important was the acceptance 
by the villagers of the leadership roles of the toiv'n minister and the 
chief contributor to the church funds Later study showed that with 
out the reinforcing effect of these leaders, the pluralistic ignorance 
in the small community was quickly dispelled, with genuine re 
versals in accepted patterns of behavior concerned with religious 
taboos 


This early community study which emphasized the individual 
as a unit of measurement was restricted in scope and in technical 
thoroughness A social psychological field study employing more 
detailed measurements is Newcomb's research on Bennington Col 
^2ge, a self contained college community (17) Measures of objec 
ttve role in the college were obtained from ratings of a cross section 
of student judges who selected the most extreme individuals in each 
class on each of 28 characteristics related to community citizenship 
Subjective role was measured by the individual s own view of her 
relationship to the community, including her awareness of differ 
ences between self and others Individual prestige was ascertained 
through nomination of students, by their fellows, as most worthy to 


represent the college at an intercollegiate gathering Finally, a series 
of attitude scales was administered to all students and ivas repeated 
for some of the students during their college careers The attitude 
scales dealt with public affairs which were central to the ^alues 
of the community The results showed that, as students assimilated 
^o the college communit}, they took on the values of the group 
he acknowledged leaders showed greater effects of such group 
membership than the followers Thus there wns good correspond 
cucc between the ^ttltudcs of students s'alued by the community 
3nd objcctue role played by the student Moreover, subjective role 
mdicatcd that withm the objective pattern there were characteristic 
personality modes of adjustment which were the resultant both 
®f the present sitintion and of past methods of relating to one's 
fellows 
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These four studies indicate that the conflict between the an 
thropological and the quantitative approach can be resolved, though 
the differences in methods should not be glossed over The anthro 
pologist or sociologist who has familiarized himself thoroughly with 
a culture or community, who has lived in it, observed its people 
talked with them at great length studied its history, and immersed 
himself in all available materials can give a picture of the function 
mg group as a whole He can make insightful interpretations of its 
social processes And this type of study provides a great deal of 
information about a community or a culture with a remarkable 
economy of effort For example, Harbison and Dubin, in their case 
studies of union management relations, produced a remarkabl) 
informative picture of the significant variables and their interre 
ations (8) Similarly, Dollard (6) and Davis, Gardner, and Gardner 
advantages ot the anthropological 
would ™d class A measurement approach 

level of n?d f T""* *■' "> 'he same h.gh 
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be readily resohcd b) utilizing the anthropological approach as the 
initial stage in a field study This stage can utilize to the full the 
adrantages of seeing the situation as a ivhole and of attempting 
to grasp the fundamental relationships From this study can come 
the insights ivhich can furnish the hypotheses for later, more 
detailed, quantitatue study In fact, in the Festinger Schachter 
Back study of a housing community preasely this procedure was 
employed (7) In the first stages, informants and informal partiapant 
obser\’ation were used, as well as observations of important group 
meetings Then, in the later stages of the study, systematic interview 
tng of all housewives was earned out, and sociometnc techniques 
'vere used to discover communication and preference patterns 


STEPS IN THE CONDUCT OF A FIELD STUDY 

It IS important, then to turn to the steps in the conduct of a 
field study The following model cannot be fully realized in every 
study Moreover, speafic studies often dictate their own proce 
•iures But there is some advantage in breaking dowm an inves 
bgation into its major processes Thus, the follovnng phases can 
be examined for their rclev'ance to a contemplated study (I) pre 
hnunary planning, (2) the scouting expedition, or the anthropologi 
•^1 short cut, (3) the formulation of the research design, (4) the 
pretesting of research instruments and procedures, (5) the full scale 
operation and (6) the analysis of materials 


^^eliminary Planmng 

Ideally, the field study should start with a period of research 
P inning m which some tentative decisions are made about the 
scope of the study, Its general objectives, and the timetable of its 
*^ges As a general rule, exact formulation of research design is left 
a later stage, when the results of the scouting expedition are 
^'ailable Often one purpose of the field study is the obtaining of 
® better knowledge of the significant variables rather ilian the final 
testing of ^ well formulated theory Even where the field study 
^ 3 follov\ up of other research however, it is important not to 
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freeze the design before tlie scouting expedition It is difficult and 
sometimes impossible to know what measures are feasible in a given 
field setting without a firsthand exploration of the situation 

It IS well to be alert to the general temptation to envisage 
the study too broadly and to make an unrealistic appraisal of what 
can be accomplished within the time and budgetary limits of the 
project The scouting stage can be more valuable if there is some 
major focus and some restriction of area Another temptation 
g nst which planning offers protection is the tendency to accept 
auc hec“™7“'' ” easy accesstbth^y 

Thes^are 1 ““Pe”'-" ‘--om a few key people in it 
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■I'e research findings and methnuT' re both substan 
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nuensively and ^\hlch specific processes aie most fruitful foi the 
study But if some of these questions can be posed in advance 
of the exploratory scouting, much time can be saved 


The Scouting Expedition 


The scouting phase of any study is the period of informal and 
relatively free investigation m which die field workers try to get 
as thorough an understanding of the important forces in the situa 
tion as the) v.an During this period they either Uve in the group 
to be studied or make frequent trips to obser\e it at first hand The 
scouting expedition is, thus, not a pretest in which already formu 
lated instruments are given a field trial It is essentially exploratory, 
tvith the objective of finding out what the significant variables in the 
Situation are likely to be and what types of instruments may have 
to be constructed to obtain measures necessary in the final study 
The advantages of employing more than one investigator for 
ihe scouting study are obvious Not only is the single investigator 
limited by time pressures but his own biases need to be checked 
Moreover, with a team of field workers supplementary and com 
plementary skills can be utilized 

Although freedom for the investigator to follow inteiesting 
leads and to utilize his own ingenuity in obtaining information is 
the very essence of the scouting stage of a study, this is not freedom 
m an absolute sense of random or aimless activity Enough is known 
about social groups in general so that we have some knowledge of 
the types of things to look for in most social situations For example, 


though the Lynds (15) envisaged their study of Middletown as 
exploratory, with no attempt to prove or disapprove any set of hy- 
potheses, they did assume that there were broad categories of basic 
social behavior that should be observed— getting a living, making a 
home, training the young using leisure in various forms of play and 
3rt, engaging m religious practices, and engaging m community 
activities As a rule, the broad framework of types of beha\ior to be 
^udied will vary in relation to the purpose of the investigation 
^'ertheless, there is mudi to be said for having some sjstematic 
P^3n in mind for scouting purposes to ensure the type of coverage 
‘nformation which the nature of the stud) may require 
The following framework is suggested as a possible guide for 
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obtaining broad coverage on important aspects of group function 
mg It will not of course, have equal value for all types of field 
studies, but it does reflect many of the types of variables i\hich social 
psychologists and sociologists are finding significant in the under 
standing of the specifics of group behavior 

1 A description of the total structure under study with respect 
to the major groups and subgroupings 

2 The central lalue systems and goals of the total system and 
Its various groups 

3 The nature and types of conHicts and points of tension both 
with respect to the total structure and with respect to a single 
group 

4 The formal and informal structure and the way in which they 
are interrelated 


5 The accepted pathways to group goals including 
a the logical relation between paths and goals 

le remoteness of paths from ultimate goals or the 
rmaTegl"*’^'” “ e-up. acivtty and its 

' O' two mam paths and 

on'“etf "T »' t'tocture under stud, 

n tne society or larger uui, of which ,t .s a narr 

XZ;: '-t.n sCctiire and its 

■» The patterns and cta'rs Tc"'" 

•tructure and the substructure, 'It' 

It IS also helpful for the BeU . . 

ways of gathering information trained in informal 

procedures whidi can be follorved Th number of practical 

from study to study The foHnT«/ their usefulness will vary 

mind by field workers when thev nn should be kept in 

•tito a community, a group, or 
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an industrial plant to cairj oiii the anthropological part of the 
nuestigation 

1. Contacts should not be limited to a narrou segment of m 
formants People ,nc \er) limited in their information by their daily 
social roles They not only lack knowledge of the activities of others 
but they are biased by the positions they occupy Hence it is impor 
lant to be in communication with some person from ev cry important 
subgrouping and every important type of social role This has been 
the classic erior in diplomatic intelligence m the past, observers 
m a legation post have moved among people like themselves, rep 
resenting the upper social strata of the country they are studying 
In general, when we enter a new community we tend to seek out 
people very much like ourselves So, too, does the inexperienced 
field worker Even interviewers with specified quotas of respondents 
to obtain will, if not checked by certain controls, bring back an 
undue number of interviews with people of the same religious and 
socioeconomic characteristics as themselves Hence, the field worker 
should be alert to the problem of obtaining a vMde coverage of 
tnformants Opinion studies which have not met the requirements 
of precise sampling have often been surprisingly accurate because 
they have obtained a fairly wide spread of respondents from all 
important types of groupings 

2 Informants who themselves have a wide range of contacts 
should be utilized The person who by virtue of his role or his 
personality has a high rate of contact may have especial usefulness 
for the field worker People involved in communication activities 
maintain many contacts and often have information which does 
uot appear m their official reports Newspaper reporters, for exam 
pie, can readily describe the pattern of leadership in the community 
^ud identify the hierarchy of infoimal political bosses 

3 Informal leaders, as well as the people in positions of formal 
leadership, should be located and consulted The account of the 
formal leader always needs to be checked and supplemented with 
^'bat the informal leader can contribute Not only is the infonml 
leader often in possession of facts and interpretations not knov\n 
^o the official, but he may also be in a better position to express 
freely what he does know For example, in one field study of union 
management relations one of the men at headquarters, holding 
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no official position could describe union policy very frankly He 
admitted that there was some fat in the present group piece rate 
that the company was trying to change on the occasion of the intro 
duction of a new assembly line The real issue was not the attempted 
change but the basic intent of the company in wanting change 
If the company wanted minor and reasonable modifications, that 
uas one thing, but if this meant a neiv policy of pushing the union 
to the wall, then the men would fight against the most minute 
change with all the resources they could muster And until they 
could get a better hue on basic company intent they intended 
to do some experimental skirmishing with the company on the issue 
of new standards The responsible union officials would not express 
this point of view so explicilly, but later events showed that it did 
represent union sentiment and union policy 
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5 Information from respondents slioiild be assessed m reia 
tion to their social role and position, their group membersliips, and 
ilieir personal activities Hence, it is important to get as much 
information about the informant’s prionlj of group allegiances, his 
position in the potver structure of the groups to ^\hich he belongs, 
his major roles, as ;vell as Ins own aspirations and goals in life 
One principle that generally holds true for hierarchical structures 
ts that the people at various levels in the hierarchy are generally 
niore sensitive about the actions and feelings of those immediately 
above them than of those below them Advancement up the ladder 
depends upon an upward orientation and an ability to relate oneself 
effectively to one’s superiors Hence, a foreman in a plant may know 
tnore about the way of thinking of his immediate bosses than of his 
own men 

In addition to motivational biases, which need to be known 
before information can be assessed, is the factor of the amount of 
knowledge which the informant can be expected to possess on the 
basis of his contacts and experiences This t\as the reason for the 
previous emphasis upon finding respondents who are very active 
3nd who enjoy wide contacts m the community One caution that 
needs to be stressed has to do with the positional lag in information 
"^he man who has moved up in a hierarchical structure knows 
through his own experience and tlirough the important contacts 
he had in the local group, about the lower levels from which he 
has come In his new role, however, he has generally lost these points 
of communication Nevertheless, he often feels he can talk accurately 
3boijt his former level of association Similarly, the general in the 
army will talk authoritatively about the problems of his men 
because twenty years ago as a lieutenant he was close to them and 
Understood their way of thinking This informational lag is less 
true of organizations based upon functional representation, where 
the leader must report back to the group which elects him This 
procedure immediately forces contact between the leader and the 
people below him But if the organization lacks functional represen 
tation, the information which high level leaders may give about 
the lower levels may be irrelevant and inaccurate 

b Ideally, it is desirable to spend considerable time in partici 
p3nt observation Reports of informants and information derived 
rom secondary sources need to be supplemented with living in the 
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community participating in its activities and constantly observing 
what people actually do m speafic situations There are frequently 
practical difficulties with participant observation, since it adds 
greatly to the length of time the scouting phase will take But there 
IS no good substitute for having field workers actually live in the 
community and perform some of the roles they are interested in 
studying Empathic understanding of the problems faced by the 
people under study cannot be obtained fully through hearing about 
experiences from others or even from direct observation Thus, if 


are making a study of an industrial plant the field workers will 
have a mudi better account of the situation if they can actually 
spend some lime both in the manager s office and on the production 
line In the absence of experiences from participation, the field 
wor er should spend as much time as possible in direct observation 
He should attend meetings of organizations and observe people 
Situations There are interesting discrepancies between 
say m isolation and the way in which they behave 
when they are under group pressure 
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iti\e belling fraternity was giving odds of 55 to I that Dewey would 
be elected 

The field ivorker needs to be aware of this type of public fiction 
It can exist where a belief has become so prevalent that few would 
dare to challenge it It can also exist in the area of taboo subjerts, 
such as sex The Kinsey report, although not based upon representa 
live sampling procedures, does raise the question of the extent to 
which accepted beliefs about sex practices are public fictions 

8 Full records should be kept by field workers Part of the 
discipline of the in\estigitor is rigorous note taking and the setting 
aside of at least two periods daily during which the notes are elab 
orated into a full report No matter how excellent his memory, 
the worker cannot reconstruct from his notes his original observa 
tions after a lapse of time without some losses in completeness and 
accuracy This is especially true in the field situation where a con 
Slant succession of similar experiences may maximize retroactive 
inhibition and inaccuracies 

9 Initial impressions and global judgments should not be 
discarded Although detailed documentation is the goal even during 
the scouting period, it is nonetheless true that this is also the period 
'vhen maximum play should be given to over all impressions As the 
Gestalt psychologists have demonstrated so effeciivel), the human 
niind does grasp things as a whole But this type of whohstic percep 
tion tends to be neglected in our scientific eilorts at precision In 
the scouting stage, however, field workers should be encouraged 
to record their initial impressions These first judgments can be 
surprisingly useful because the situation can sometimes be perceived 
to Its main outlines at the very start As the exploratory work 
progresses there is a tendency for the details to obtrude themselves 
Therefore, there is some point for the investigators to try a summing 
Up at stated intervals to make them see the whole picture again 

10 Available records and secondary sources should be studied 
carefully, and the operational procedures for deriving such records 
should be examined Not only are such existing inate-ials of great 
value in the understanding of the situation but they sometimes 

be used as measures of variables in the larger study For exam 
plc a field study of an industrial situation ma) well want to inquire 
fully into productivity and other records maintained by the com 
pau) being studied When the report is made of the scouting expedi 
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tion. It IS not enough to know that productivity records exist on 
individual workers It is essential to know what these records are 
based upon, to what degree the products ity of the worker is 
set by the pace of the machine or the assembly line and to what 
degree by his own efforts, how comparable productivity records are 
for individuals performing different tasks, how stable are the pro 
ductivity differences reported over time, etc Unless the operational 
meaning of these measures is known, it is impossible to construct a 
research design which will utilize such records 


The Formulation of the Research Design 

.u scouting exploration become available, 
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ment in the group In this second type of study, "we ivould develop 
detailed measures of these independent variables and would make 
exact predictions for the productivity of work groups varying in 
group standards and solidarity We would also specify the conditions 
which have to be held constant for these predictions to be realized 
Since these conditions may not be held constant directly, we would 
measure them to achieve some statistical control over their effects 
Ideally, the testing of hypotheses is more suited to laboratory 
experimentation, and exploratory discover} to field studies and 
bur\e)s This does not mean, however, that field studies should con 
fine themseUes wholly to exploratory procedures The scoutingstage 
can often be used as the more purely exploratory part of the investi 
gation, and some degree of h}poihesis testing can be employed in the 
larger operation to follow Moreover, there are occasions when the 
field approach can be used for very important hypothesis testing, 
as m the "natural experiment" (see Chap S) But it is nonetheless 
true that the great strength of the field type of study is its inductive 
procedure, its potentiality for discovering significant variables and 
basic relations that would never be found if we were confined to 
research dictated by a hypothetical deductive model Thus, the field 
study and the survey are the great protection in social science against 
the sterility and triviality of premature model building 

It IS possible, of course, to combine both exploration and hy 
pothesis testing in a single field study One major set of hypotheses 
can be investigated at the same time that other materials are gath 
®red for exploratory purposes This has the advantage of protecting 
the study from failure if inconclusive results are found with respect 
to the hypotheses The exploratory materials then become the safety 
factor The disadvantage of this compromise is that it attempts to 
combine tivo studies in one investigation, sometimes to the detri 
tttent of both 

E\en an exploratory study should be so designed as to provide 
^ definite information as possible for a set of research objectives 
lere are at least two levels of exploratory studies At the first 
''Cl is the discovery of the significant variables in the situation 
the second, the discovery of relationships between variables Even 
the first level it is important to delimit the area to be studied 
to introduce controls into the data collection process Explora 
studies which do not set limits for themselves have limits im 
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posed by various practical matters, some of ivhich are not realized 
bv the investigators 

For the exploratory study aimed at the discovery of variables 
rather than relationships factor analysis is often urged as the best 
method of finding out the unitary and independent factors m the 
situation From a design point of view, much more is known about 
the assumptions of factor analysis on the side of statistical treatment 
of materials than about the assumptions on the data collection side 
of the process The tendency in the applications of factor analysis to 
social settings is to throw all sorts of measures of various degrees 
o precision and \alidity into the hopper of factor analysis and to 
depend upon statistical sophistication to grind out meaningful 
entities Controls on the data collection side, such as measures taken 
under standardized conditions and an adequate sampling of situa 
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Thus It resembles hypothesis testing in resting its case upon the 
relationships discovered rather than upon the precise use of mathe 
matical techniques The major difference between such an explora 
tory study and the hypothesis testing investigation is that in the 
former there are no specific predictions of relationships based upon 
theoretical derivations The researchers do have hypotheses in mind, 
but these are not precisely formulated In a study of class structure 
m a community, for example, we may start vMth the general assump 
tion that a significant motivating factor in class identification stems 
from the economic role which the individual pHys But we may not 
be prepared to specify what we mean by economic role, or what 
other roles may account equally well for psychological chss iden 
tification Therefore we plan our research so as to study the many 
possible types of economic role, including the part the individual 
plays in consumption, in the technical aspects of production, in the 
social aspects of production, etc ^Vitliin the broad frame set by our 
lesearch objective, we hope to find some significant relationships 
Or, in a study of industrial morale, we may be concerned with the 
in plant factors which are related to worker satisfaction We shall 
include all the important aspects related to the job and the plant 
from wages and working conditions to type of immediate super 
vision and congeniality of fellow workers Then, in analysis we hope 
to find significant relationships between worker satisfaction and 
some of these in plant factors 

In this sort of exploratory study, the design should be so con 
structed that measures are available for all relevant dimensions of 
the area under investigation but the study should be confined to a 
limited type of problem It may be that a ivhole set of variables, 
which have been omitted as not belonging to the area under investi 
&3tion have more to do with the dependent variable than the fac 
tors which are studied For example, economic role conceivably may 
not be as important in class identification as sociometric personal 
preferences or the length of residence of the family in the com 
tnumty, or the number of ancestors who fought in the Revolutionary 
War But it is a mistake to believe that one study is going to be able 
to account for all the variance in complex social phenomena It is 
nitich more effective to take one central sec of variables and investi 
S^te them as thoroughly as possible than to tiy to study the universe 
m one piece of research This is a widely accepted principle among 
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research workers save when they evaluate the research of other peo 
pie Then objections are often raised to studies because they omit 
many significant causal determinants Thus for example to the 
extent that the California studies of prejudice (1) are concerned 
with the relation of personality dynamics to discrimmatorv atti 
tudes It IS not legitimate to criticize this objective because group 
membership economic interest etc are not investigated We make 
progress in science not by trying to solve all problems at once but 
by going at things one step at a time 

The best opportunity for the use of hypothesis testing is on the 
c Sion Q t e natural experiment The difficulty with the use 
of hypotheses field stud.es ts the tnab.l.ty to deiernttne causal 
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of the AVar Department was able to measure some of the effects of 
contact under these conditions of group survival, although the study 
was not a natural experiment in that its measures came after the 
event Nonetheless, when a planned change is known, it is feasible 
to formulate hypotheses in advance and to take continuing measures 
and observations of the on going change The full advantages of the 
natural experiment remain to be exploited 

It IS true, of course, that to the extent that psychologists are 
allowed to participate in a planned change by being fully informed 
and by being permitted to take detailed measures, the way is open 
for them to influence the planning of the change Thus ue have 
a bridge between the natural experiment and the field experiment 
An interesting illustration of the possibilities here is found in the 
Curie Trist study of the rehabilitation of returned British prisoners 
of war (4) Many of these men were having real difficulty m readjust 
*ng to their old roles in the family and community Hence, camps 
'vere set up in which a group of men could live for a period and 
niaintam something of the norms and roles "which they had devel 
oped as prisoners of war But contacts with civilian life were grad 
ually established to facilitate adjustment to the civilian world This 
natural experiment was studied very carefully and was to some 
extent carried out on the advice of psychiatrists and psychologists 
The major advantage of the natural experiment over the lab 
oratory or planned field experiment is that the manipulation of 
variables is much more powerful The real world can and does pro 
duce role ceveriiU, dta&uc vx uorais, 

^■evolutions, and group conflict in a fashion impossible in the lab 
oratory The design difficulties in the natural experiment however, 
much greater than in the laboratory experiment We generally 
^ack a control group whose comparability to the experimental group 
assured Hence provision should be made in the design for obtain 
•ng measurements on as well matched a control group as possible 
This does not guarantee that the experimental and control groups 
'\ill be truly equated on e\erything but the independent sarnble 
It does, houeNer, increase the piobnbilities tint the predicted results 
if confirmed are valid 

A second provision to make is the detailed measurement of the 
degree to which the independent variable is manifest m the sub 
kroiips under study If for example, in the Arm> experiments we 
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uere studying the effect of contact, we would want to know how 
many Negroes were actually introduced into a given combat unit 
In other words, we need a measure of the amount of the inde 
pendent variable which is independent of the measure of its effect 
A third provision should allow for continuing observations during 
the course of the change process Such observations can be helpful 
m the interpretation of resulu because they may show many proc 
esses which intervene between the initial change and the final 
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setting out such tables, the investigators are bound to discover com 
plexities o£ a variable vhich need more detailed measurement 
and qualifications o£ hypotheses in relation to special conditions 
The less the study attempts to make specific predictions, the less 
detailed the preliminary sketch o£ the tables needs to be 

Regardless o£ the degree o£ hypothesis testing o£ the field study, 
the design should exploit £ully three o£ the natural advantages o£ 
such investigations The first advantage is that the field study tends 
to continue over a period o£ time, so that it is possible to maintain 
continued observation Thus it follows that the timing of certain 
variables may be ascertained We can do very little in ex post facto 
analysis, where we are dealing with variables which are untimed 
If we find in industry that supervisors who follow democratic 
human relations skills have sections with larger productivity, we 
do not know with any assurance which is cause and svhich is effect 
We can assume from general psychological knowledge that it is 
likely that the skills were not produced by higher productivity, ut 
we are much further along scientifically if we can time the occur 
rence of these two variables In this case it would mean following 
given supervisors as they are transferred from section to section 
Even where the time period of the field study is not itself very long 
It can afford chances to check on the timing of factors through the 
consultation of records and the use of the memories of a number 
of respondents Unless the design clearly specifies the types of such 
measures for the timing of variables it will be difficult in later 
analysis to pin down such information 

A second advantage of the field study which should be utilized 
is the opportunity for the direct observation of interaction and of 
social relationships We make inferences about social process and 
social structure from surveys but in the field study we can observe 
these factors more directly If, m our study of the college com 
munity, we are interested in the effects of group membership in 
various organizations we should not be content with getting the 
attitudes of these group members in isolation We should have 
obseners at group meetings to record how people actually behaie 
the group situation 

A third advantage of the field study is the important resource 
of going beyond measures obtained from a single instrument The 
correlations from a single measuring instrument may be influenced 
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b) some halo m the application of the instrument Leaders who 
report good communicatton practices with their followers may also 
be the leaders who report good communication practices with their 
own superiors This may mean not a true generalit) of relating 
onese to others in the organization but optimism or convention 
ahty in answering the inteniewers questions A field stud) permits 
o taming of reciprocal perceptions and interdependent reac 
ions from dififerene groups of people whose behavior is interrelated 
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houses destroyed in the respondents town as ascertained by the 
field investigator Thus, the interview could deal with various prob 
lems of morale during nar, such as confidence in leadership 
equality of sacrifice, etc, without the respondents realizing the 
major objective of the interview Hence, relationships between 
morale and degree of exposure to bombing could not be attributed 
to the halo effect of the interviewing instrument If in designing 
a field study, we limit ourselves to responses from the interview for 
all our measures of the independent and the dependent variables 
we neglect the unusual potentialities for methodological advance in 
the field approach 


The Pretesting of Research Instruments and Proceduies 

The elaboration of the research design of the study should 
contain the specifications for the measures required These measuies 
call for such instruments as interview schedules questionnaires 
behavioral scales, and forms for the gathering of mfonnaiion 
Wherever the research objectives permit instruments thai hive 
already been standardized in other studies should be employed 
This use of common instruments would facilitate the comparison 
of findings from study to study Nevertheless it is still true that in 
most investigations for some time to come we shall need many new 
instruments developed to suit the objectives of the study 

It is essential that every new instrument be pretested before 
the full scale field operation Such pretesting has three purposes 
(1) to develop the procedures for applying the research instrument 
so that, tor example, the scale or schedule can be used effectively 
with respect to the time it takes to administer, (2) to test the word 
mg of questions so that they are suited to the understanding o t ic 
audience, and (3) to ensure as far as is practical, that the specific 
questions or observations are really getting at the variable for 
which a measure is needed The pretesting of instruments is too 
often limited to the first two objectives since tliey pose readily 
soluble problems There is always a genuine danger that the tliinl 
objective may be slighted in favor of tlie interesting question t ic 
observation easy to make, and the general ease of administering the 
study The novice easily becomes discouraged by the v-ariable that 
's difficult of measurement consequently he shifts, often without 
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realizing the extent of his shift, from the research objectne to the 
easil) obtainable infoimation 

Another difficult) iMth developing measures for the variables 
daignated in the design is the lack of a good criterion for deciding 
nhen ne hate a good measure kVhere it is not feasible to validate 
the measure against an objective criterion, the following procedure 
IS recommended The instnimenl should be tried on a number of 
mses and the materials obtained should be coded bv a number 
judges are instructed about the meaning of the 
ihp make independent assignments of 

amnnnr^'r'n records to ordered categories expressing 
Zoml S 1 “^servers can Z a be 

they can actual situation and then check on whether 

in which the ^ needed and whether they agree on the waj 
usedtton L"“ T ‘>pe of procedure is 

■s bringmrout Te'r'a 'r ‘he question or device 

- -t:^"ng“:he v:X'’'Thir"r' ^ 

check, but It creates a nre<„ " ’ "ot a true validit) 

toward developing the ‘'^'“hdtty and moves a long wa> 

And the major codes for cv.l 

tional materials should he ‘he interview and observa 

The final cod.„g'“ 0“ sZld ““I 

and revtstons of th«e majol cat^or'^”" 

'o this form of pr'etetme^a? studies, the time and attention given 
that are scekintr to h” ^ '"^^equate And jet, to the extent 
Sceater critical sign,r,can'cTlhan''tr'°“'"P,' P°‘"‘ 

objectives into opcraiionaJ translation of the research 

>ield no results and the brilliant theorj vmII 

'raste of time if ue fail to d i statistical analjsis will be a 

ranablcs VMth vshicli we operational measures of the 

often true that less time and Unfortunately, it is too 

than upon almost anv othf*r if spent upon this problem 

, J‘ o .mjiorlan. /o ZZ '““-h process 

cethires upon a population as sim i of measures and pro 

will be studied If the field stn i* j* possible to the people who 
communitj. u « possible to prct«r I? at a large group or 

innuencing the results of the Hr 'tisiruments without undulj 
ger study to come It is advisable. 
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in a siiuatioii in whicli the populaiion is very laigc. to determine 
the sample in advance so that the pretesting can be carried on 
outside of the sample to be used. 

In the pretesting of instruments and procctlures, it is not essen- 
tial to obtain a representative sample of subjects but it is important 
to try to include some of the main types of people who will be 
included in the final study. A pretest of an instrument on a few 
subjects from only the upper education brackets will not anticipate 
the problems to be met among those with minimum schooling. 
This, of course, applies more to the last stages of pretesting. Often, 
when we arc first trying to develop a measure, we arc less concerned 
with replicating the final field situation than we are with getting 
insight into the fundamental character of a given variable. 

Tlie pretesting of measures on populations very similar to those 
which will be used in tlic latter study is necessary to determine 
both the specific form of questions and of observational codes and 
the types of measures applicable to specific groups of people. It has 
been demonstrated that the written questionnaire is mucli more 
widely applicable than was formerly assumed. It has ihe^ great 
advantage over the personal interview of anonymity, which in 
some situations can compensate for the knowledge gained from 
actual personal contact with the respondent. Moreover, there are 
great economics in the use of written questionnaires as against 
interviews. But it is still true tliat written forms are much better 
adapted to well-educated groups. The losver skill levels among 
Workers not only may have little formal schooling but they may 
have spent very little time with paper and pencil materials since 
their school days. It is more natural for them to express their ideas 
orally than in writing. The exact line to be drawn in the use of 
interviews a« against written questionnaires can be determined 
through pretests. It is often advantageous, when employing written 
forms on a large scale, to draw a subsample of respondents for per- 
sonal interviewing. Thus the biases in the one method can be 
checked by means of the other. 


T'he Full-scale Field Operation 

Ideally, almost all the research problems are solved before the 
study goes into the field, but unfortunately these problems do not 
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remain solved during actual field operations The content of i 
research instrument, veil pretested, takes on a new meaning be 
cause of its relation to events vhich base suddenly changed The 
scales de\ eloped m adsance for behavioral observation prove useless 
for bringing out the critical points m the group meetings under 
study It IS veil, therefore, to have a research team in the field 
which has some balance between ingenuity and soundness of judg 
ment ft is often necessary to improvise and make changes m pro 
cedures during the course of the field work But if ingenuity m 
s,»h or®,r" unchecked, the research team will lose 

the nrr ' research objectives In the actual field Situation 

the L fi IS strong to meet the practical needs of the moment at 
the sacrifice of long range research plans 

siderabK frnm P^fsonnel for the field operation differ con 
for the field ^ large scale survey The tasks 

for the production varied and often more difficult than 

field tork:r‘‘“br::ie‘':rxrhr 

communu, but much more gfoups in the 

leaders and ranking officials (10) 

of the Survey Research PpI. * Human Relations Program 
mental organizations industrial and govern 

the leadership levels of imno^l* equipped to deal with 

"orkers needed some exnerien ""'eetigatmn Field 

fonal and administrative problLT'lnr'’ 1'^'’^''“®'' 
siderable graduate inm.J Individuals who had had con 

and considerable experien? P’>'^°'“Sy =>"<! the social science*! 
to be the most effective wori..*" a? government proved 

need to differentiate in the li held studies of this sort 

include a few people >\ho * * * their field workers and to 
leaders at something approarh!!^ capable of meeting outstanding 
S.nce most fielcf s.uZt"®H " 
op to ensure comparability of k **' controls should be set 
different periods of the study P i obtained during 

m the study may have been ^ ^^*cr perioil 

ring since the start of the siudv o t, ^ natural social events occur 
with earlier subjects These tw ^ reports from and interaction 
procedures The effects from th^ ^^'1 different 

m Y Itself suggest as rapid a 
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measurement of an entire group as possible and then passing on to 
another group This is also the more economical method But the 
effects of events in the world suggest that \\e do not interview all 
of one type of respondent at a single time period The difference 
between groups may then be a difference between the people before 
and after an event This conflict can be solved by concentrating 
heavily upon one group at a time but presiding for small sub 
samples of all groups for the major time periods of the study, even 
if it means some reobservation of the first people studied 

The field study must face the important problems of obtaining 
cooperation from the many individuals and subgroups in the struc 
ture under investigation In a survey, it is generally not necessary 
to give much time or effort to obtaining the cooperation of re 
spondents, since most individuals will give an hour or more to the 
interviewer, if he will come at a time convenient to them But in 
a field study where the research workers may spend weeks and even 
months in the same community, where they may reinterview the same 
people, seek access to privileged information, and attend all types 
of meetings, the matter of obtaining community or group support 
takes on unusual importance (10) The following procedures are 
recommended as deserving consideration 

1 There is a real economy in going to the very top of the 
structures under study to obtain the cooperation of the ranking 
leaders This is especially important if we are dealing with a 
hierarchical structure, as in industry, where lower levels are always 
dependent upon their superiors and are too insecure to risk wel 
coming investigators from the outside It is very probable that the 
matter of cooperation will be referred up the line sooner or later, 
snd there is a much better chance for a favorable decision if top 
leaders are consulted at the outset In addition, the top leaders are 
often more likely to understand what the research is all about and 
are more open to conviction The greatest resistance is frequently 
found among the petty officials, not only because of their insecurity 
but because of their general limitations 

2 The easiest entrance into a community or a social structure- 
oamely, coming m as the ally of individuals or groups who have a 
special interest to exploit and who see the research as a means to 
their ends-should be avoided Thus, the partisans of a specific 
reform m a community or the executive vice president of an indus 
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inal company seeking information on the delinquencies of his sub 
ordinates may welcome the researcher and offer him support The 
alliance with such special pleaders is neither ethical nor wise The 
researcher s aim should be to enter the situation in the common 
interests of all parties, and his findings should be equally available 
to all groups and induiduals The cooperation offered a partisan has 
two disadvantages It may result in a lack of cooperation on the part 
o ot er people, and it may exert undue influence upon the re 
search objectives 


It may be of course, that this ideal is too difficult to achieve 
in practice, since the researcher may sometimes have to accept help 
^here he a>n find it But the broader the basis of h.s support, the 
naliM^ community see that social science has poten 

finHin ^ ^ ihose who want to avail themselves of its 

itener-d operations can be To ensure this type of 

advisors snrm'""''' '“pporb n IS often advisable to set up an 
■n thecommumty"® “'O diverse tnteievts 

should be vive'' ^bout the study and its basic purposes 

Lade“a? 17 f '*■' community, as well as to the 

investigation This ""‘■’om Piaji'dicing the results of the 

IS desirable to cnvA ^ amount of information which it 

field experimentation^^ ih field research and 

It extends over more thL a'f^ d’' " 

a conspiracy of a impossible to maintain 

cteases^heVreafand vvm"""^ -crecy merely m 

does not want his potential "7 cumors Yet the researcher 

specific hypotheses and obje'ctivefT 

an explicit statement at •» ^ common solution is to present 

pies of Items which are n ^ S^^^ral level with one or two exam 

■n presenting an explanr™’ .'“u'”' 

the accepted channels fL "e*’ “ “‘dire 

■n the communit, ,f „.e in the various groups 

the study may become idennST?!.'* <*annel, 
that channel In industrial stud mterests associated with 

should appear both in the co account of the coming research 

newspaper. mpany house organ and in the union 
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It should be recognized that any field study entails some usks 
The introduction of observers prying into various aspects of com 
niunuy life may not be relished by all groups in the community 
Certain questions may be offensive to some people And at times the 
climate of opinion may be such that people will cooperate neither 
m launching the study nor in allowing themselves to be interviewed 
The risks should be known from the scouting explorations and 
calculated in advance Jn general, however, these risks are exag 
gerated m democratic societies People in general like to be inter 
viewed and are receptive to the idea of cooperating with social 
scientists The general experience is that the usual practical ob 
Stacies in getting information which obsess the armchair critic are 
not so difficult to overcome as the problems of research design 
4 Another aspect of long run cooperation is the ethical stand 
3fd of the research worker m keeping faith with people who have 
helped him This involves a religious preservation of the anonymity 
of the respondent and a thorough carrying out of the spirit and 
letter of the obligations incurred in the study Every provision 
should be made to protect the identity of the respondent Analysis 
of materials should not be carried to such a point and case mate 
should not be quoted in such a fashion as to permit idenii 
ncation of specific individuals If people are told that the identifying 
marks on their questionnaires will be removed, these marks should 
^ removed Research workers have rightfully made a fetish of this 
preservation of the anonymity of respondents and the assumption is 
at the vs pvvw 

and Willingness on the part of the subject to become known 

Another, more subtle problem concerns the specificity ^\ith 
leh findings are reported for particular groups and subgroups 
if individuals are not identified, the reporting of results for 
groups can place them in positions of advantage or disadvantage 
general rule is that the cross breaks should be reported so as 
0 show the general relations rather than to show the incidence of 
ypes of values and behaviors in small subgroups Such general find 
should be available to everyone The researcher cannot of 
bourse, control the uses to which his results are put, but he should 
to provide equal access to his general findings for all groups 
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The Analysis of Materials 

In a ^\ell planned study m which hypothesis testing is the 
j^incipal objective, the major codes will have been developed in 
the pretesting of instruments In an exploratory study, many of the 
codes must still be developed after or during the period of data 
though the study is exploratory, it is important to 
degree of conceptualization rather than 
maior type of response (see Chapter 10) The 

richness of s^’tne of the 

In general fh ® not cover all phenotypical details 

.he“ful'.r" '''' deta-Ie^the cod.ng 

to be recombin j ^"ttlysis For analysis purposes, the data have 
f observattons neglected 

all measures These"trrilVt* “ “'’ni" ''‘^‘1“='’'')' distributions on 
obtaining these fre ^ tables have several purposes In 

mecham® 1 pTo ceHr^sif 

totals and subtotals r u /ting material to Hollerith cards (all 

ttoss breaks vvh.cLhould be rufTf ‘"/°™ation about the 

"0 teal spread of responses them is n ftaqiency table shows 
relate It with another variable Tf 1 attempting to cor 

tions of the amount of ho t t j example, our coded observa 
m group show that 90 Derr^m' *7 against members of the 

score, our chances for obtainm ^ ^ ‘"dividuals had a zero hostilitj 

small indeed If the total m 'vith other variables are 

re vvork ,„th such undigcrem,aTed',rrV' “ 1’°“'’’'' 

absays consider the size of ih ^'slributions The analyst must 

table before going ahead correlational 

of straight run tables decision Through an inspection 

of steps uitlim a * made about the combination 


of steps uitliin a category or* made about the combination 

gories to comprise an mdev ®*5out the combination of cate 
been too refined for the niimber*^^^*"^ may haie 

indicates that the seven decree ? straight run table 

to fixe or to three Or a numht tnay have to be reduced 

tion in group activities may h^\ u ‘^®*’®tit measures of participa 
■nallv ihe frequency disir.biuio “ ®‘"gle indev 

^os to be examined in relation 
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to the correlation measure to be employed in that correlation 
assumes definite properties with respect to the distribution of the 
measures being correlated 

The decision about the cross breaks to be employed is already 
determined in the hypothesis testing study In the exploratory study, 
It IS \Mse not to try to run every variable against every other variable 
but to decide on the basis of all the information available what the 
most promising relationships are likely to be 

In general, material from the observations and interview 
responses in a field study or a survey lend themselves more readily 
to correlation analysis than to analysis of variance Analysis of 
variance is better suited to experimental studies where we have 
approximately equal groups assigned to known values of the experi 
mental variable The procedure of the breakdown or cross break is 
a correlational procedure, and every table reporting a cross break 
IS a correlational table Frequently, correlations are not computed, 
since the comparison of groupings within the table can show the 
relationship though the precise amount of relationship requires a 
correlational computation Although it is a great advantage to be 
able to state the degree of relationship for the entire table by a 
single correlation coefficient, there is often no statistical rationale 
for such a procedure Correlations make assumptions whicli the 
data may not justify To compare the significance of the difference 
between two subgroups in the table demands fewer assumptions 
about the nature of the data 

The exploratory study calls for ex post facto analysis m which 
've want to make the most plausible interpretation of our findings 
even though the interpretation can never be definitive The usual 
procedure here is to try to develop a relationship or to check on its 
spunousness by holding factors constant through the use of triple 
breaks The most ingenious and systematic use of this method is 
exempliHedmtheworkofP Lazarsfeld who with P Kendall has 
presented a codification of such analysis procedures in Continuities 

Soaal Research (13) Three main types of elaboration are de 
scribed which differ not so much in essential method as m giving 
ibe analyst a framework for thinking about the relationships m his 
One type of elaboration is to control for spurious factors In 
this case, we have discovered a relauonship between two vTinables 
but we want to make sure within the limits of our data that it is 
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not a function of a third factor which would explain away the 
relationship Thus, a relationship rvhich we discover between job 
satisfaction and skill level of the job may be due to the fact that 
igher skill jobs are better paid A triple break will allow us to hold 
wages constant for different skill levels in relation to job satisfactions 
nstead of trying to explain away our findings as due to some 
spurious factor, we may want to show that some other factor, whose 
imingwe can establish, has intervened to produce the relationship 
unZ ^ ^ Research Branch of the Army, the 

hkel though in general less critical, were more 

bettir H ^ should have been deferred than were the 

ma hyP°‘hes.ze. that a vanable 

about defermp^t'"^ i, factor of education and feelings 

the factor of "i reason for the relationship-specificalh, 

supposedly comin” f' Tfut poorer educated men svere 

among the better^cH”™ ®^'’“P^ "'here deferments were higher than 
dSent m 2 f “'"“■'“We on the rate of 

"as really with defenue^"'* '"P'^ break, the correlation 

honship 'b:™met ■"’"P— " 

elaboration is that^the test '“'"“el Pomt m this type of 
some acmal variable that has m[e“e"ned 

find the conditions^under^'^H**^ specification-that is. trying to 
We may find m an mH * relationship will be accentuated 

small relaiionshiD her study, for example, a positive but 

foreman and the morale^Tt. human relations skills of the 

for expecting the relation h We have theoretical reasons 

specify the conditions unZZ Z higher, and hence we seek to 
"'e postulate, therefore th correlation should increase 

are effective m dealmc LtZluZ where the foremen 

will be higher, since, m addii ^‘^pcriors, the morale of the men 
ran accomplish things on then "L't “"‘''‘'“anding their men, the) 
in which we compare the m ^ ^ means of a triple break 
good human relations praciic"” ' c' "orkers under foremen mill 
foremen less skilled, for dilfe^!'!! morale of workers under 
effectiveness up the Imc we ca ®Srces of conditions of foreman 
’]^|“re of the relationship " our specifications about 

the tcscarchcr can'unhrc nosM!^"™ '"n'-enient ways in iihich 
jsis in making his data )ield 
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the most meaning In tins process, the findings are used to suggest 
hypotheses, which are then examined and tested further through 
manipulation of the data The form of statistical manipulation in 
the three types is the same, but the logical and theoretical implica 
tions differ It is true, of course, that this type of hypothesis testing 
after the fact is limited by the data already gathered and is there 
fore no substitute for the controlled experiment Nevertheless, such 
analysis can greatly increase the probability of the soundness of the 
interpretation of the data 


The meaning of the data should not be speculated about ivben 
It IS possible to test the speculations in the data themselves Often 
by cross breaks and by holding factors constant the statements which 
the researdier sets forth as interpietation can be checked against 
his own data If confirmed, it can be stated not as speculation but 
as a finding within the limitations of his study 


Field studies frequently pose the problem of the proper N to 
tise for computing the statistical significance of a difference It is 
common to use the number of individuals in a subgrouping as the A' 
'vithout regard to the possibility of clustering effects Where v\e are 
dealing with a cluster, the proper to use in computing the sta 
itstical significance is the number of clusters, not the number of 
individuals Suppose, for example, we are studying the effects of 
different types of adult education practices upon the acceptance 
and use of local public health clinics There are three discussion 
S^oups of 12 members each in which a participation method was 
^sed and three groups of 50 members eacli in which a lecture method 
'vas used In comparing the effectiveness of these methods, it is not 
justifiable to regard our Ns as 36 and 150, respectively We do 
have 36 separate and independent manifestations of the dis 
Mission method since in any one group the use of the method uas 
^lored by the discussion leader and the composition of the group 
the true N here is 3 Similarly, in the lecture method there 
not 150 independent effects of the metliod but rather thiee 
^ajor applications for it It should be remembered that the prob 
6 error of the difference is based upon N s vs Inch are independent 
Measures of the effect in question 

The field study is particularly susceptible to cluster effects In 
e laboratory we can set up our experiments to guard against this 
iculty In surveys we can sample m random fashion to avoid 
But in a community or a group under field study our 
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cases are often pockeleti in homogeneous subgroups, where the 
clustering effect must be orefully considered 


the place of field studies in 
PROGRAMMATIC RESEARCH 


In building a science of social psychology it is desirable to 
a e advantage of the different settings in which our phenomena 
ur an o approaches which utiliie the particular advantages of 
The field study is unique in enabling us to observe 
hand processes m their natural occurrence On the one 

other hand '"•dcrsianding to survey findings On the 

hypotheses for experimenter rich insights and 

laboratory from rvdomr' 'h' 

to do wtth » system of concepts which have little 

Inirl n V" P'“P'' behave 
must operate m'^ln ®''enis in a natural setting the field study 
so called alien factors ma "'‘e‘'eeting variables in which 

gists iiould agree about 'h so poorly defined that not all psycholo 
Hence a. the premm , ‘ “ alien factor 

gamed from wLkmg bmf m 11,^1'*^''’'“’^' 

model from phssics^nn h 'aboratory, where the restricted 

"here the 100^' “^.!':''' ■*='- “"d ■" the field Situation, 

ences is the prototype employed m the biological sci 

studies IS to cil/attent!rn*^t**'”f potential usefulness of field 
ception Early experimental work psychology of per 

‘gnored the svholistic qualiti« J Ii at scientific rigor, 

cecded m overanalytic fashinn Psychological field and pro 
ments of consciousness The « with highly artificial ele 

the early experimental uork "^^aJ^ness was to be observed in 
*n the emphasis upon social processes in the laboratory 

the complexities of social important to analyze 

there is always the danger that o social relationships, but 

funciioml properties and 

artifacts than functional cntii.r. elements which are more 
cnui.es F,eld studies, by their close con 
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tact with on-going social events, can serve as a check 

omission o£ significant variables. In a long-range progra 

search, there should be a two-way interaction between _ 

tation and field studies. The findings fiom field studies '"ay > 

questions which need for their solution the more rigorous methods 

of experimentation. Conversely, the experimental 

a basis for the formulation of some of the researc o j 

the field study. If the relations which hold within e 

also prove of significance in the field situation, it is an i 

that the laboratory has not moved in too artifactual a d'rection. 

There is an important intervening step, however, between the 
field study and the laboratory experiment-narnely, the fie p 
ment. The natural experiment, previously discusse , i 
change which takes place without any action by the 
It just happens to be an interesting chanp or im 
The field experiment, however, is a social chan^ °”^'„-n„Iation 
the researcher. He is instrumental in effecting t e ma ip 
of a set of variables in a life situation. The field experiment s the 
logical connection between the field study and the laboratory ex 
periment. It follows the logical procedures of the laboratory mo 
closely, but it nevertheless deals with factors operating in the fie 
situation and thus, like the field study, is more concerned with 

“global” variables. . ^ 

Field studies can furnish the essenua! 
make a successful field experiment possible. The most P 

in following up the leads from the field study is experimentation i 
the same situation, so that there can be some actual manipulation 
of the factors which appeared to be causal deteiminants "> “c hew 
study. It is to field experimentation, therefore, that the nex p 
is devoted. 
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CHAPTER THREE 


Experiments in Field Settings 

JohnR P French, Jr. 


The desirability of widening the scope of experimental 
social psychology through experimentation in "real life” settings 
‘■^cognized for some time Lewin has pointed out that 
though It appears to be possible to study certain problems of 
society in experimentally created, smaller laboratory groups, we 
shall have also to develop research techniques that will permit us 
to 0 real experiments within existing ‘natural’ social groups In 
my opinion, the practical and theoretical importance of these types 

o experiments is of the first magnmide” (30, p 164) 


'VHAT IS A FIELD EXPERIMENT^ 

such methods progress has been made in developing 

me ioToM^' "'i experiment is not yet a well developed 

of r dated ^ <■'«-= - -nety 

operattonal re ea ; evLluat.on research, 

m field settings In diTcmsing 

method tn socmlogy, Greem,oS 7 r 8 "'rr“ 

(.) .he pnre e.;l™e;r:^.r;s“h:!’e?a eT;h:^t; 7 r 7 To 7 , 

« hatTc'lW t?^'’ “"“"‘^hed eteper men . or whal 

no hate called the natural experiment (see pp "78 79 ), ( 3 ) the ex 
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post facto experiment, in ■which the investigator tries to trace ba^ 
irards from an effect which has already occurred to its causes, (4) e 
trial and-error experiment, which seems to refer to a sorts o na s 
by laymen of new forms of social behavior, (5) the contro e o ser 
rational study None of these five categories describes the field 
experiment, although the major dimensions along w ic t e ca e 
gories differ are fairly adequate lor such a description According y, 
the meaning of the field experiment and its re ation to o 
methods can best be clarified by considering variations of three 
aspects the design of the research, the setting, and the purpose 
The essential factor which distinguishes the field experimen 
from the more common “field study’ is the design of tie researc 
The field experiment involves the actual manipulation of conditions 
by the experimenter m order to determine causal relations, w er a 
m the field study the researcher uses the selection o su jects an 
the measurement of existing conditions in the field setting as a 
method of determining correlations As we shall see ater, one o 
crucial methodological problems of the field experiment is 
ways of manipulating the independent variable It is^ this i y 
which has led to the use of the ’ natural experiment, m which the 
researcher opportunistically capitalizes upon some on going “^nSJ 
and studies their effects in an experimental design t ese na 
changes have already occurred by the time the social scientist arrises 
on the scene, it may still be possible to gather sufficient data after 
the fact to fill out the design of a crude ex post facto experimen 
In the field experiment, the manipulation of the independent sari 
able IS not left to nature but is contrived, at least in part, by the 
experimenter, thus, the design is planned by him beforehand 
The wide variety of truly experimental studies that have taU 
place in field settings vary in purpose all the way from the deselop 
nient of social psychological theory to the immediate so ution o 
some practical social problem Sometimes both purposes presen . 
2s m the experimental forms of ‘action research tv ic i ave 
dual purpose of bringing about social change and at t e same tun 
contributing to basic social science (7, 8, 27, 39) But most applied 
tcsearch has the major purpose of obtainiug tacts and attitudes o 
evaluating methods which will be of immediate value >" 5°''" » 
tome specific applied problem, although theoretical developme 
•nay be a minor purpose (44) Such practically oriented research is 
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the most common type of field experiment For example, studies 
evaluating the relative effectiveness of two types of political propa 
ganda or of two teaching methods or of several advertising appeals 
attempt to get fairly immediate answers to practical problems of 
politics, education, or advertising without attempting to apply any 
general theory In fact, there are very few field experiments with 
a strongly theoretical orientation, yet there is a sufficient stockpile 
of theory and knowledge in social psychology to warrant many such 
field experiments Most of all we need experiments whose purpose 
is to test the applicability to real life situations of the known scien 
tific laws or hypotheses specifically developed in controlled labora 
tory settings Throughout this chapter we shall try to emphasize 
the scientific and theoretical purposes of the field experiment 
The setting for a field experiment is some real existing social 
situation in which the phenomena to be studied are commonly 
oun y implication it is not an “anificiar situation created in 
a research laboratory This distinction, however, is not nearly so 
cearcut as u seems at first glance, for it is not important whether 
e social phenomena occur in a building called a laboratory rather 
social institution The relevant 
a iTn T studying real and studying 

to thelr' "■“"■"g of •Lt.ficiar as applied 

behavior^ Ti ^"°Fu '“‘’oratory seems to be that their 

Block have do f '“u' '' role Block and 
“rnrulre t "’«'<«<’ class subject almost invar, 

m^rerirm rerat”r'‘‘'n"“'“‘'‘’" a sub 

behavior is role deternitned it"i's 

one role cannot be generalized In a obtained with 

tional research In addition thp h without addi 

experiment ,s highly "" ^ JhT 1 

tuted m order to control ^ procedures msti 

involves the creation of n tretjuently. this simplification 

by their past histo^ofthrp^LP: 

hold for such restricted situaLns mav ’ "““I® 

to the more complex settings of real hfe changes 

IS not subject to such artifiaahw and “ Sc'd experiment 

ol generalizing to real hie situations 

s nat this is not always the 
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case, however, is well illustrated in the famous Hawthorne experi- 
ments (37). From a methodological point of view, the most 
esting finding was what we might call the “Hawthorne effect. In 
order to manipulate more precisely the physical factors affecting 
production, the experimenters had set up a special experimental 
room for a small group of girls who were wiring relays. This wiring 
room was separated from the rest of the factory, and the girls work- 
ing in it received special attention from both outside experimenters 
and the management of the plant. Careful studies of this wiring 
group showed marked increases in production which were related 
only to the special social position and social treatment they received. 
Thus, it was the “artificial- social aspects of the experimental con- 
ditions set up for measurement which produced the increases in 
group productivity. The distinctions between “artificial” laboratory 
experiments and "real" field experiment are, therefore, matters of 
degree: one field experiment will vary greatly from another m the 
complexity of the social setting, the controls introduced by the 
experimenter, and the roles played by the subjects. 

For the purposes of this chapter, then, we shall define ^ field 
experiment as a theoretically oriented research project in which the 
experimenter manipulates an independent ^-ariable in some real 
social setting in order to test some hypothesis. 


PLANNING A FIELD EXPERIMENT 

The idealized sequence of steps in planning a scientific experi 
"lent might include, first, the selection of a problem on the basis 
of theoretical considerations and the formulations of precise hy- 
potheses, and then the selection of some appropriate methodology 
^nd the creation of an experimental design. No such simple sequence 
» usual in planning a field experiment, because the social scientist 
does not ha^e free access to the whole world as his laboraior>' or 
tfic power to carry out most of the experimental manipulations he 
"^>gbt conceive. Accordingly, he must proceed somewhat opportun- 
‘^ically. Frequently, he is first presented with an opportunity to do 
Research in some specific setting, so that the choice o pro ern an 
“cthod will come aftenvards. In most cases it ssill be sme to 
'“"'Wer simultaneously the selection of a problem, the scicct.on 
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of a field setting, and the designing of the study, since all three of 
these factors are highl) interdependent The kind of control groups 
that are needed, for example, will depend on the problem selected, 
and the possibility of obtaining these groups will depend on tshat 
is available in the setting This interdependence should be kept m 
mind as we discuss, in sequence, selecting and formulating a research 
problem, selecting a setting, and research design 


Selecting and Formulating a Research Problem 

SELECTING THE PROBLEM The research problem and the re 
search methodology should be selected for their appropriateness to 
each other It is not easy at present to make generalizations about 
t e m s of problems for which the field experiment is an appro- 
priate methodology because there has been too little field expen 
mentation m social psychology Yet one can see certain common 
types of problems m the work that has been done Perhaps the 
problems for which a field experiment has been 
It ' methodology would be the studies on "how to do 

techtiinnfl«^ experiment has been used for studying a variety of 

method, (4^9 r'laf’the 

part,ciDa„no /m ■ group decision (28, 29) and 

gfnda ( 1 , 20) ®t'cTn a°U P™*’" 

brinffinw oK » u Pfihese cases, one or more methods for 
othef"vocdr.h? Already existed prior to the research In 

procedures which nffacnveness of change 
Also there hav k purposes other than research 

the existing formal or in'for'''^^"* miperiments which made use of 
pie, studied leaders^ u social structure Jackson, for exam 

made use of ex.,ting’’leldm“and 
the influence of milftary leaden lam'r, 
of sociometnc regrouping on ‘*“= 

Social structure has also been 

Field experimentation seemfal'so'tX""™'' a 

to use with a problem nr ni,- ° appropriate method 

laboratory-fcr examnir.t r™" ““dy ■" “’e 

e.fectiven^sofpo'S;r“»i' 

Probably a field experiment if used^n^” " 'mass education (42) 
d in many of these cases not so 
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much because it would be impossible to devise an experiment in 
the laboratory but because such artificially created phenomena 
would be too weak or would not be dynamically equivalent to the 
real thing. Thus, Annis (1) collaborated with the printer to have 
an editorial “planted" in the university daily so that the subjects 
would read it with their normal set without knowing that it was 
an experimental stimulus. Had they known, it is doubtful that the 
experiment would have been dynamically equivalent to the usual 


political propaganda. ^ ... 

Conversely, one of the more important determinants in selecting 
a problem for a field experiment is the fact that it is possible to 
study by this method only those hypotheses in which one can manip- 
ulate the independent variable or produce a change. Here it is 
most important to keep in mind the rule: “Start strong. Except 
where experimental methods are already well developed, the experi- 
menter should attempt to maximize the variations in the inde- 
pendent variable or the differences among the experimental treat- 
ments. This is necessary both because our methods of measurement 
in social psychology are often so crude as to be able to reflect only 
rather gross changes and because the experimenter, at least at the 
i^eginning, is quite likely to produce far smaller changes t an e 
auempts. There are a variety of determinants of the changes that 
«n be produced, such as the role of the experimenter, his knowl- 
edge, and his skill. These will be discussed further in a later section 


of this chapter. , 

This same rule would require that we study major rather than 
minor independent variables. If our dependent variable is deter- 
mined by several factors, we cannot hope to determine experi- 
mentally the effects of the minor ones until we understand (and 
ran control) the major ones which account for most of the variance. 

In other cases, the field experiment seems to have been the 
preferred methodology because it reduces the problem of genera - 
■ration and application of results. Much of the "operational re- 
■rarch” in the armed services and in other organizations is of this 
character. Such applied research usually evaluates the effectiveness 
“f operational procedures. Many a study of college sophcimores ha 
h«n criticized on the ground that the same results would not hold 
'or soldiers or industrial workers, but such an objection cannot be 
"rado for a field experiment using soldiers or industrial workers as 
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of a field setting, and the designing of the study, since all three of 
these factors are highly interdependent. The kind of control groups 
that are needed, for example, will depend on the problem selected: 
and the possibility of obtaining these groups will depend on what 
is available in the setting. This interdependence should be kept in 
mind as we discuss, in sequence, selecting and formulating a research 
problem, selecting a setting, and research design. 


Selecting and Formulating a Research Problem 

SELECTING THE PROBLEM, The research problem and the re- 
search methodology should be selected for their appropriateness to 
each other. It is not easy at present to make generalizations about 
t e kinds of problems for which the field experiment is an appro- 
priate methodolo^, because there has been too little field experi- 
mentation in social psychology. Yet one can see certain common 
ypes 0 problems in the work that has been done. Perhaps the 
argest category of problems for which a field experiment has been 
he appropriate methodology would be the studies on “how to do 
^ ^ experiment has been used^for studying a variety of 
2 * adv^tising techniques, training 
DaniciDiiiil' group decision (28, 29) and 

ganda^l 9 m ’ ‘^poup therapy (10), political propa- 

brineinp-'ah ’ cases, one or more methods for 

ote lr^ / "> 'ho research. In 

prccedmes effectiveness of change 

Also thpr. I, ^ 1 . purposes other than research, 

the existing fn. several experiments which made use of 

X sXl lL V Jackson, for exam- 

■nade me of exis^^^eld””;,'^ -changing fommen (21). Snyder 
the influenr#* nf .v,,-!-. , . their groups m experiments on 

of sociometric regrouping Tn 

Social structure iSs also been nsed^’^^™"'"’' f 

Field experimentation seemfalToTT”“‘* ‘ ’a d 

lo use with a problem or phenom ' '‘PP">P'''ate method 

laboratory-for examnl^s fU ®™enon too difficult to study in the 
effectiveness of political prom,^ n"®'’'® ’’“‘’its (28) or the 

Probably a field experiment education” (42). 

expenment k used m many of these cases not so 
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accounting for the distinctive characteristics oE role playing— namely, 
the degree of involvement in the role. Other charaaeristics were 
neglected, and attention was focused on this one variable. Accor 
ingly, the experiment attempted to create three degrees of invo ve 
ment in the role in order to test explicit hypotheses concerning t e 
effect of such involvement on perception, feeling, and participation 
in discussion. 

As a problem or a social technique for bringing about a^ c ange 
becomes more conceptualized, it also becomes clearer that it invo ves 
several hypotheses and that tnere is a real choice as to how g o a 
or how analytical to be in stating a hypothesis. Whether to work 
with a global syndrome or with a narrowly defined variable poses 
a problem about which there is considerable difference of opinion 
among different disciplines of the social sciences and ^ 

ferent theoretical approaches. We cannot discuss it fu y lere. 
Rather, we shall attempt to point out certain aspects especially 
relevant to the field experiment. 

Consider Fig. 1 as a schematic representation of the causal 
interrelations among the experimental manipulations, the in^de- 
pendent variables to be manipulated, and the dependent vana es 
'vhose changes are to be measured. In this example, our genera 
hypothesis predicts that the more democratic the behavior of the 
first-line foreman, the higher the production of the employees (the 
arrows indicate the causal relations). Our independent variable ot 
foreman behavior might be manipulated, for example, y training 
the foremen {A in Fig. 1). But how shall we define "democratic 
behavior” for the purposes of our training course? ^his patten 
behavior consists of many parts or dimensions whic mig t 
treated as separate variables; for example, the amount of treedom 
or closeness of supervision (/„), the amount of participation per- 
tnitted in planning (/A, the arbitrariness of discipline (/e), and 
much the foreman does to satisfy the needs of his employees ( j). 

But these aspects are interdependent so that a ciange in 
'dll produce a change in any neighboring aspect, (f'' yg. , 
mterdependence is represented by a heavy boundary and high inter- 
dependence is represented by a less heavy boundary 
boring regions ) Now, in analyzing the data froin a , . 

field study of production as determined by the behavio 
foreman, the investigator is free to choose any part or any combina- 
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subjects In short, by study o£ the specific setting and problem 
which IS to be solved it is possible to avoid the problem of gen 
eralizing from one situation to another 

On the other hand some problems are appropriate for a field 
experiment precisely because they require generalization to a real 
situation These are the hypotheses proved in laboratory experi 
ments but not yet sufficiently studied m field settings We ha\e 
already indicated that this should be an increasing source of field 
experiments 


Finally, of course, many problems appropriate for field experi 
ments have come out of field studies Often these are theoretical 
problems in which a hypothesis has been too confounded in the 


nonexpenmental studies or where a correlation has been estab 
hshed without the possibility of determining the direction of causa 
tion The Survey Research Center at the University of Michigan 
as conducted several field studies of industrial organizations in 
which It has found that organizational units with high productivity 
have more democratic supervision Because either could be reason 
a y interpreted as the cause of the other, the Center put the matter 
to experimental test (34 35) In a field experimenl the locus of 
control was actually manipulated toward more de 
groups and toward more centralization 
mcreasp c set o£ groups The hypothesis would predict 

second treatment and decreases m the 

possible tn^ experimental design xt is no longer 

cratic supei^wmn^^ productivity might be causing the demo 
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lion of parts of foreman behavior as the independent variable this 
includes the freedom erroneously to treat plus as a single 
variable instead of choosing some combination more closely corre 
spending to the actual interdependence (for example, and or 
choosing Zg alone In a field experiment, on the other hand, there 
IS less probability of error in deciding how analytical to be in 
formulating the independent variable 


EXPERIMENTAL INDEPENDENT DEPENDENT 

MANIPULATION VARIABLES VARIABLE 



DEMOCRATIC BEHAVIOR 
OF FOREMAN 

The causal relations among experimental manipulations 
independent variables and dependent variables 


fact, I IS hiphlv in, j atomistic by varying Z* alone If. in 

produtt a change Thut'* ‘‘ 

varied both vanables-a resuit^h'T''™“‘“ ^ 

Structure of the situation Tf il corresponds better to the real 

'’anables the exocrimpm **** Pleasures of all four independent 

-u,d 

P dent variable in a way which would 
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yield better predictions. By attempts actually to manipulate a \ari' 
able in the field setting, it seems that the social scientist has a better 
chance of finding out what parts necessarily hang together. He 
partly avoids the error of a too broad or too narrow definition of 
his variables. This is important, because it is one way of making our 
concepts conform to the essential nature of the phenomena we study. 

Fig. 1 illustrates also the advantage of the field experiment over 
the field study in determining the direction of causation. The e 
experiment can distinguish between independent and depen ent 
variables, whereas the field study can only establish correlations 
among variables. In this example we have assumed that increasing 
the level of production will also change certain parts of the demo- 
cratic behavior of the foreman— i.e., it vvill reduce the closeness o 
his supervision. Since we have also assumed that closeness of super- 
vision causes lower production, we have here a circular social process. 
In order to analyze fully the nature of this circular process, one 
would have to conduct two field experiments in which each of the 
two variables was manipulated as an independent variable. Prob- 
ably a good many real social processes contain circular causal rela- 
tions of this type. It is important to analyze these phenomena 
experimentally. . , . 

In formulating a problem for a field experiment, one s ou ^ 
state explicitly the hypothesis to be tested. As in laboratory expen- 
®ents, the more this hypothesis is derivable from a general theory, 
the better. It is also important to formulate hypotheses about a 
Other possible causes of the same dependent variable so t at one 
«n try to hold these other factors constant in the design of the 


experiment. 

RESEARCH OBJECTIVES AND PRACTICAL OBJECTIVES. In t le e^ 

‘'Oh of a problem for a field experiment, the range of possible 
ptoblcms will be determined partly by the relation of the theoretica 
Pbtposes of the scientist to the practical purposes of the subjects 
^"■1 their institutions. Sometimes there will be commonality of 
SM's: oltcn there will be dillerenccs. These dillerences m goals 
"tran that the activities in which the researclier must engage 
toach his scientific objectives may be seen by the practitioner 
'"terfering with his goals. Accordingly, these dilTering purposes 
’ bmit on the freedom of the experimenter to choose anj problc 
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of interest to him The less interest the practitioner has in the 
development of science, the more limited will be the researcher's 
choice 


One tempting solution to this conflict, which may frequently 
have bad effects on the research, is the attempt to study too many 
problems at the same time— that is, the social scientist collects data 
m order to test certain theoretical hypotheses and at the same time 
tries to collect additional data to satisfy several of the practical 
needs of the organization in which he is working This results in 
a dispersion of effort and a lowering of the quality of the research 
Another solution which reduces the scientific value of the research 


IS to compromise on the degree of generality in formulating the 
pro em for the purposes of science, maximum generality and 
a stractness of the concepts are desirable, whereas the practitioner 
o en wants nowledge about his specific situation with all hs 
Solution to this problem seems to be to 
\ possibilities of conflict as well as the possi 
of ma ° interest and then to select for study a problem 

'he practitioner 

(see pp 123/)°^^ ^bout this cooperative lelationship later 


Selecting 0 Setting 
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researcher will find some people who are favorably disposed toward 
social science research and willing to cooperate Contacts can e 
made most easily with those organizations which feel the 
research to help solve their own problems Particularly for a e 
experiment, which will entail certain new and unforseen changes 
m the organization, it is best to contact organizations where there 
IS both a motivation to contribute to science and a realistic expecta 
tion that the research will benefit the organization This suggestion 
implies that the researcher must show his willingness to help t e 
organization No statements, however, will be as convincing as 
past services to organizations or past field studies which have a 
beneficial results 

In selecting a setting for a field experiment, the researc er wi , 
of course, look first for a social situation which contains the phe 
nomena he wishes to study This is not a simple matter of determm 
ing the presence or absence of the phenomenon, but rather a ques 
tion of discovering whether it occurs m a sufficiently pure form or 
strongly enough so that research is feasible Actually this may reqmre 
considerable knowledge and understanding of the setting For 
example, a researcher is ho wished to study the strength o 
standards about performance m work groups as a function of the 
cohesiveness of the groups selected bomber maintenance creiss in 
the Air Force This seemed an ideal setting because there isere so 
many identical groups Later he discovered that these groups isere 
too small, that their performance isas difficult to distinguish from 
the isork of other maintenance men, that the crews isere not 
fact, identical, and that probably the group standards isere gen 
erally i\eak 

the influence or the experimenter Among tlie man) settings 
"hich may be relesant to the problem he wishes to stiid>, there 
'sill be important variations in the degree to which the c\pcrimcntcr 
has freedom and poiscr in the setting or organization to be studied 
of all. he needs the poivcr to manipulate some independent 
'anable or to iri out some technique of social change in a con 
‘rolled design Tlie poiscr and influence required for such e\pcri 
mcnting vill depend lerj much on hois upsetting tlie cxpcrinicn 
procedure is to the organization , 

T^oth his poiser and h.s skill m carn-ng out m cxi^nmemal 
“faimcm m a field setting ma> depend upon the researchers r 
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tionship to the practitioner If his own skills or the power of his 
role are insufficient, the researcher may be able to use a skillful 
practitioner as the actual manipulator of a variable Not only does 
such a relationship increase the possible range of successful expert 
mental problems, but there is frequently a great value in separat 
ing the action role from the role of data collector For example, in 
experiments on therapy, training, and the like, there is probably 
much less bias if the researcher is not also at the same time the 
therapist or trainer Thus it will often be desirable or even essential 
or t e researcher to select a setting m which such help and coopera 
tion from a skillful practitioner are available 

It IS also desirable that the experimenter have the power to 
comro ot er factors which might confound the experiment In most 
new setting, this seems to be possible to only a limited demce An 
stitute for actual control is knowledge, especially fore 
to factors For example* if one wishes 

essemnUn on production in a factory, it may be 

of the exop^*"^**^*** * ^ factory from laying off workers in the middle 
It 01 to h"' *>' ™P°«>ble for the tesearcher to 

durina a time ? *** P'®" experiments 

Freedom of '*' factor will not disturb them 

selectme a setnnn consideration in 

not only m them w^illinan '’‘P”""'"' Organizations vary widely 
but also in the extent to wh n 
available An equally essentiaffr®”? 

dom to publish the rL i. r l fn^ the scientist is the free 

nirrictiLTiEs in tu * without undue censorship 

ments in areas in which 7 h^ Conducting field experi 

some extremely difficult ena a'l ” “F'^^'h^ed social conflict raises 
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virtually preclude gaitring cooperation from the other (22). Thus, 
the scientist may be forced into a position which will be seen as 
“taking sides.” and therefore access to important data will not 
be permitted. It seems important for the scientist to maintain an 
independent role even if it means that he does not study severe 
conflict situations (17, 24). It is especially true that too much con- 
flict within an organization can make impractical the launching 
of a field experiment which requires a great deal of cooperative 
effort (22). . . 

The extent of recognized problems in an organization an tie 
felt need for help in solving these problems present a similar dif- 
ficulty in selecting an appropriate setting. ‘ If the organization is 
faced with a large number of acute problems it is not li 'e y t lat 
its leaders will feel able or willing to divert efforts from ‘fire fight- 
ing’ to the investigation of the causes of their difficulties. On the 
other hand, the management which feels that it has so ye ^ a o 
its problems has little incentive to engage in a research re ations^ ip 
of any kind" (22, p. 66). Engaging in a field experiment requires 
more change than permitting a field study. Hence, an orpnization 
nsually must perceive the problem to be more acute in the n™” 
‘ban in the latter case in order to-engage in research. Jaqiies (2j). 
"ho places heavw emphasis on the therapeutic aspects of his ivory, 
studies only those problems concerning which the client reques 
help. 

■niE NEED FOR SCOUTING. Taken together, all the const era 
‘ions so far discussed imply the need for a considerable hotly o 
infonnation, some of it rather subtle and tiifficult to acquire, e ore 
a wile choice of selling can be made. Unless the cxpcrimcii cr 
aheady has direct knowledge of the situation through previous heu 
'oearch in it. it will be necessary for him to do a considerable 
amount of ••scouting" (see also Cha,.. 2). The experimenter should 
'■'Sit die setting so that he may directly observe behavior and inte 
ym"- a number of people in different positions. It is ■ 

'miwrtaiit to discuss their conccptioti of the cxiieriment ly.th those 
"ho will help to make the tlecisions about the r«earch am , 
Pm'ible. to talk with those who might have conihetmg an udey 
losvaui it. Particularlv where there is potential conflict > 




-rj'WMU" parties, it mav take longer to ••h"- 

'm^aicl, shnnUi inocectl than it takes to get the necessary mloima- 


j»ft agreement ttiat the 
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tion about the setting In his intensive therapeutic study of a factory, 
Jaques spent three months working out the arrangements for the 
study and the relations between the research team and various parts 
of the factory (24) 


EXAMPLES A few examples will make clear the several aspects 
of field settings discussed above and will illustrate some of the roles 
and relationships that experimenters have established We shall 
choose examples illustrating three mam types of relationships— first, 
where a single experimenter is an outsider who belongs neither 
to t e organization being studied nor to a research organization, sec 
on , where the researcher belongs to a large independent research 
organization, and third, where he is a member of a research unit 
within the organization studied 


th ^ study of leadership (21) is an interesting example of 

u a skillful experimenter can obtain even 

ivn« n« ^ 6 p of a large research organization The experiment 
an M a thesis while the author was a graduate 
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the union) resulted in enoneh (and with 

lation was carried out and ^P”" Prejnct so the manipu 

respect to transfers, despite oressin"™" 

The mam reasons for success ® for making changes 

lo be the experimenter’s skill ,n exol'"™® ““Pomtion seemed 

mg meetings and in leading ErounH?"'"® 

g group decisions, and the fact that the 



Experiments in Field Settings 


113 


company got as a by product, without cost, the methods it needed 
for evaluating foremen 

An example in which the relationship is between an inde 
pendent research organization and a client company is foun ^ ^ 
work of the Survey Research Center at the Universit) of Michigan 
This research agency had a quite specific contract with a large 
insurance company to conduct a field experiment (see p 
effects of the location of control and regulation (34, 35) This con 
tract specified the independent role of the research organization 
and guaranteed its right to publish the findings Since t e researc 
team had no authority in the company, it could carry out t e 
mental manipulation only through the help and cooperation o 
management Such a manipulation as delegating contro an regu 
lation, of course, required not only a long series o meetings o 
arrange the new duties and responsibilities of supervisors ut 
a long series of training sessions to prepare them to ^ ® 

responsibilities In this process the research team use ot me 
of the company and outside specialists To a 
large, complex, cooperative venture could succeed ecause 
pany and the research organization had already coopera e 
field survey during which good relations had been esta 
the research organization had been able to discover in 
initial state of the phenomena to be studied m t e exp * 
namely, the actual locus of control and regulation In ^^d tion 
however, they had acquired considerable information about the 
internal relations within the organization an ten 
acuteness of various problems j 

Quue a difterem relat.onsh.p between the 
the industrial setting was involved in some field experi 
the Harwood Manufacturing Corporation (9) ere ‘ j 
had an inside role as personnel manager and irector o p 
research Even the first studies in this organization coi 
experiments, because the top management was a\o ,31 

research and sophisticated aLut it This was dt.e to — 1 

'ttct that the prendent of the companj had a doctorate PJ^ 

The author’s position in top management, and strong support from 
'he president: permitted freedom to do eri 

some possibility of holding conditions „[ /onp 

"tent-tor example, during experiments on the ell b 
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decision on the level of production it ^vas possible to preient super 
visors from exerting any influence on the production of their em 
ployees— and where it w.as not possible actually to control confound 
ing factors it was usually possible to know about their existence 
in advance On the other hand, this inside position meant that the 
experimenter v,as subjected to stronger influences of practical con 
company goals and policies Although this inside 
ro e e to better understanding of the dynamics of the factory as 
well as mside knowledge of events before they occuried, it also 
c ear y imited access to certain information Initially, when there 
was no union, it was rather easy to obtain a wide variety of informa 
at^ali^ipv^i^ uman relations in the plant by interview ing people 
imnn<i<i,hi the plant became unionized, it was 

seouentlv^t ° rapport with some union members and con 

of problem^ information from them For certain kinds 

outsider with an independent role 


Research Design 

pnncTpk?hfs‘rme a°s thrS^t'a Jab”' “ experiment are in 
difficuliifi , e 'or a laboratory experiment, but the 

greater in the fieId™euinr"Th*"'* eontrolling variables are often 

lems ot control ^ added emphasis to some prob 

CONTOOL GROUPS The USP nt A 
especially important m most KpW * “t^trol groups is 

retting IS both complex and because the real life 

tors at work Furthermor S'ng, with many uncontrolled fac 

these possibly confounding facrl” qu‘te ignorant about 

a supervisory trainine nro^ experiment on the effect of 

might be confounded by sudTf employee morale, for example, 
pany personnel policies etc S ^ ® ^*se, changes in com 

there were significant chances ^ Hanton showed that 

even in the control groups direction of the training goals 

with the control foremen on s f ‘earned foremen were matched 
of morale, but the method ^ type of work, and level 

of the company so that two subjects was by divisions 

mental group and two in the placed in the expen 

group Later it was discovered 
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that the major determinant of the effectiveness of training vvas the 
human relations practices of the foremans superior (the uision 
manager) which, of course, was systematically different tor the ex 
perimental and control groups This is but one example o an organ 
izational factor which can easily confound an experiment since it 
affects a whole group of subjects at the same time in the same way 
Ideall) the control gioup should be matched tMth the expenmenta 
group on all possible confounding factors, matching on t le asis 
of easily measured v ariables such as size of group age o su 1^^^^ 
etc will do no good unless these variables are systematically relate 
to the hypothesis to be tested 

Solomon Ins shown (41) that the conventional design with a 
single control group is inadequate where the pre experimen a 
measures interact with the experimental treatment and influence its 
effectiveness He conducted an illustrative field experiment on the 
effects of teaching spelling in a grammar school and demonstrated 

that the preliminary Spelling test actually interfered It t e rai 

mg The preliminarv test resulted in smaller gams in the experi 
mental group and in the conventional control group than were 
shown by a third matched control group which did not take the 
preliminary test but did receive the training Solomon also mentions 
interaction effects and effects on \ariance in studies ° ^ ' 
change Canter has shoisn similar interaction effects in a stud) o 
human relations training (5) On a pTwri grounds one 

pose that certain measurement procedures such as open en e 

views, might produce even larger interaction effects Accordingly 
n seems desirable, where possible to use an extended contro group 
design in field experiments This would involve the comparison of 
niatched croups on post measures 

coviioL TllRoi^CH MEASUREV.EXT fVherc the experimciitei 
u not able to standardize conditions the effect of confounding f-ic 
>ors may nevertheless be discovered tbrougli measurement If ic 
knows all the additional dcicrminants of Ins dependent varn e 
the experimenter can measure them and determine y ana y 
"heiher any of them were confounded with Ins experimental var. 
•tble Thus the field experiment will commonly require more ^ 
nremem of the total situation and particularly of any tm™" ’’ 
'hanges occurring during the course of the experiment If the experi 
■lent has been preceded by field studies in the same selling a t 
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on the same problem it will be possible to knou moi e of the possibl) 
relevant factors which must be measured 

REPLICATION Becausc of the difHciiIty of standardizing condi 
tions and because conditions may vary o\er time it is desirable 
where possible to design field experiments tint can be replicated 
m the same setting For example Bavclas ran a series of experiments 
on the effects of group decision on production in the Hanvood 
Manufacturing Corporation (28) These experiments took place 
over a fairly long period of time and under somewhat varying condi 
tions The present author subsequently repeated some of the same 
experiments in the same setting at a later date (15) Such replication 
in the same setting is often difficult for group experiments because 
o an insufficient number of matched groups To replicate m 
experirnent m an organization setting there must be a number of 
para e or matched units Where the units or groups are not matched 
Antw j ^ performing different functions or are differ 

organized-new and irrelevant variables may intrude them 


experiments Planning for replication of a field 
Drov,Z ‘’“■8" advantage of 

and thp r °PP°“^riity to perfect the experimental manipulation 
Zoomn , « not planned tt .s all the more 

of the evZ Pf'l'nnnary experiments to ttork the bugs o it 

be stfficTeml r‘ nnd ,o ensure that they will 

n na of Z L -n-surable dtHerences The run 

puently rnoZh P has not in fact been done fre 
mdicate the imn several field experiments would 

the necessarv P™'"''-ng ample opportunity to practice 

controlling '^moZm' T “5“?^ 
control groups and adequate '=‘P'"n'ent through the use of 

these methods first becausl they arrih'’“"“‘ 

Actually such control doe7no? Z 

in conditions it makes it nossiH. ™minate unwanted variation 
There are two ways of experimeiZZ’’' ‘h‘= had effects 

or reduce such variation Lndarvl ' '^hich do eliminate 

insulation »'andaniiration and what we shall call 
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Sometimes it is possible for the experimenter to standardize 
conditions in the sense of holding constant something w ic mig 
othenvise vary in the field setting In several studies of the effects 
of social variables (such as group decision) on production in a tac 
lory (30, Chap 9), it was possible to hold constant i^o^t of t e 
variables affecting production— the type of machinery, t e ow o 
materials, and the composition of the group (no trans erring o 
employees into or out of the group vvas permitted) On t e ot er 
hand, absenteeism and slight variations in the quality o materia 
could not be prevented The field experimenter should attempt 
maximum standardization of sucli conditions 

Insulation is less clear as a form of control, but tve shall use 
this term to refer to the elimination of certain conditions normally 
existing in the field setting In the Hawthorne studies, or examp e, 
the Relay Assembly Test Room was insulated from the rest of the 
plant both physically and socially (37) There were walls separating 
the experimental group from the other employees, an t e norma 
supervisory relation was eliminated Evidently the conditnms create 
were so different from the usual factory conditions that this expert 
nient stands midway between a field experiment and a a orato^ 
experiment A much milder degree of insulation was used in the 
experiments on group decision, on pacing cards, and on pro uc i 
hy instructing the supervisors not to say anything to t eir emp oyces 
about level of production, speed of working, steadiness of wor mg 
«te (30, p 215) Thus insulation, by eliminating some variables, 
simplifies complex conditions so that the experiment becomes more 
liVe a laboratory experiment (see Chap 4) 

In Itself, such simplification is neilher good nor bad, rather, it 
must be evaluated in terms of the same kind of considerations 
tclevant for choosing how analytical to be (see p 105) or w et ler 
to use a laboratory experiment or a field experiment 

A special problem of insulation is the possibility that one 
experimental group may contaminate mother experimental group 
m the control group Communication among groups can lead to 
serious confounding In an experiment on the effects of prrticipation 
°a production, for example, competition developed between two 
of the groups, thus confounding the effects of participation (9) 

SAFETY FACTORS Because of the danger that the expcrimcnial 
manipulation may not be strong enough-in the sense that it docs 
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not produce sufficiently large differences— it is desirable to design 
the experiment with certain safety factors” so that some of the 
values of a field study may be obtained even if the experimental 
manipulation fails For example an experimental study of the effects 
of human relations training obtained measures of changes in both 
group structure and group process (4) Even though these variables 
showed little change that could be attributed to the training, there 
were nevertheless some interesting findings on the interrelations 
of process variables and structure variables 


SIZE OF UNITS AND LENGTH OF TIME. In general, it IS probably 
much more effective to apply experimental treatments to units of 
air y small size— individuals or small groups rather than large insti 
tutions Dealing with smaller subgroups permits easier manipulation 
an provides an opportunity for preliminary experimentation and 
rep The size of the experimental manipulation should also 

nf possible by confining it to a relatively short period 

nrn>.rK 1 pcriod of time required, the greater the 

onlv tA ° unforeseen outside changes taking place One has 
vanahlM changes over time in existing data on such 

social uction or absenteeism in a factory, political and 

in a few changes can take place 

be confounded rr longer experiments are more likely to 

It IS desirable ^ efficiency alone, 

IS desirable to reduce the time to a minimum 


carrying out a field experiment 

required for carrvinw^ methods used and the skills 

trols and for buildm? and ^ manipulation and con 

relationships, because Aese necessary cooperative 

in reports of research T often described in detail 

difficult for there h:ie generalize about these skills is even more 
are not commonly , a' h of them and they 

ally, the practical toctal ntanagerTand 

seem to be more skillful than th professional consultants 

a planned change in a social ^ scientists in bringing about 
carry on, an exp^nmL.aTdes.^'’™ “ 
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Accordingly, this section will draw heavily on the ideas of 
Kenneth Benne and the staff of the National Training Laboratory’ 
in Group Development concerning the skills of the practitioner 
(36). In discussing social action, they point out. 

The behaviors of persons, g;roups, organizations and com 
munities may be thought of as held in their accustomed grooves 
by an equilibrium of forces tending to move the level of function 
mg in one direction or another These forces may be of various 
kinds and magnitudes— established status relations, laws, per 
sonal and group sanctions and standards, established competence 
m a given way of working, feelings of inadequacy, non perception 
of alternative ways of behaving, etc Change, in the sense of an 
alteration in the level and way of functioning of a person or 
group, or an organization or community, occurs when this equi 
librium of forces is disturbed According to this view, change may 
be seen as the transition of a system of behavior from one 
equilibrium level to another For example, a teacher may change 
from a manipulatory to a collaborative control relation to her 
group of students or vice versa, members of a group may change 
their paniapation pattern from one with a high percentage of 
individual centered roles to one with a low percentage of such 
roles, or vice versa; a community may change from a condition 
of corapeution between v\clfaTe agencies to a condition of 
coordinated cooperation, or vice versa, etc. . . . Planned change 
occurs when the forces holding the person or group or community 
at a given level, with respect to one or another phase of us life, 
can be assessed, when factors making for potential disequilibrium 
arc Understood, when a new possible and desirable state of affairs 
can be projected, and when the forces for effecting movement 
to this new equilibrium can be developed or manipulated to this 
end (56. p. 108), 

is also a good description of the complex process involved in 
^ planned manipulation of a variable in a field experiment. 

Our conception of how to bring about these cliangcs is thus 
Actually based on: (1) some theorizing about the process of change 
terms of quasi stationary equilibria (30, Chap 9), (2) some con* 
^Phons of scientific methodology as an effcctiv c method of problem 
(3) a methodological conception of democratic ethics, and 
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(4) some scientific research on tlie effectiveness of certain democratic 
procedures in bringing about planned changes \Vg shall not attempt 
to present the rationale for these foundations here but shall proceed 
directly to their iniphcitions for action First we shall describe two 
experimental procedures and then attempt a few generalizations 
regarding the methods and the skills involved 


Examples of Experimental Treatments 

A field experiment by Coch and French illustrates a method 
of manipulating the variable of partiapation and at the same time 
effects of this variable m bringing about planned changes 
( ) ince both authors had conducted earlier experiments in the 
same actory there was no problem of establishing the role of the 
researc ers in the factory Furthermore the experiment helped 
° the most difficult postwar problems of the factory 
products had to be changed as the plant 
nolopirai h ctvihan production there were widespread tech 
same^tarfrlLft!^^* affecting many employees Earlier studies in the 
larpp ^ shown that such technological changes produced 

hfni rr* res..«nce .0 the 

changes on the part of employees 

BreaMthrnC™'"'"' hypoihcsis that the 

Le the ‘•’c part of employees tn planning to 

The Chanef »e them rLstance to 

cu!sed ^th 1 ■" Thts hypothests was d.s 

problems that wo” irfeTusedT^th’’" b 

their nature Formnatelv th ^ ‘"’''>"8“ “ diagnose 

menters had experienced , ^ '"anagement including the experi 
on the effects of loh cha “"^‘d'rable body of past research both 
the effects of group decision'!n°"h productivity and on 

of the previous findings and of'*?®'"® O" ‘''‘= ‘’“'j 

through previous particiDaii„„ * ""aearch mindedness developed 

jointly not only a procedure for h'rnn''"? “ 

by a method of partic.pat.on but af '"® '^'^’"'o'ogical changes 

sjstematicallyvarpng the degree? design for 

v..?sr“ 's? f"'"™- "™ » — 

P ton or control group was changed over 
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to the new job accoiding to the usual factory procedure, which 
consisted of explaining, in a group meeting, why the changes were 
necessary, what the new job would consist of, and what the new 
piece rate would be As usual, this procedure was carried out by 
the production manager and the time study engineer The second 
treatment, “participation through representation," consisted of a 
group meeting, also conducted by the production manager, in which 
the need for a change was presented as dramatically as possible and 
the broader problem of cost reduction was shared with the group 
This led to a discussion of the types of changes that should be made 
m the product and the suggestion that management should make 
work simplification studies, tram several operators in the new meth 
ods, and set the piece work rates by time studies on these specially 
trained operators The new job and the new piece rate would then 
be explained to all operators, with the special operators helping 
to tram the others These special representatives were then elected 
by the group Later they met with the management to plan more 
about the new job, and they presented many good suggestions 
These special operators became so involved that they referred to t e 
new job as “our job” and ‘our rate” The third experimental 
treatment, “total participation,” was much like the second except 
that all operators participated directly in planning the new job 
and the new rates rather than participating through representatives 
Although only a few hours were involved in the meetings which 
constituted the experimental manipulations, it was clear that they 
had strong and very different effects Probably this was due partly 
to skillful leadership by the production manager nnd partly to the 
fact that he had an important role in the factory, which gave added 
meaning to his beha\ior Although no measurements of the experi- 
mental treatments were made, the measured effects on production 
showed much higher productivity with greater participation 

This principle of participation is of fundamental importance 
m carrying out an experimental procedure The production man 
3gcr and other members of management were motnaied to co 
operate in the experiment because of outside economic pressures 
combined with recognized difficulties in handling such problems in 
*be past Certainly their past experiences witli similar research on 
group decision contributed to both their understanding and their 
tupport 
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Quite a different problem of experimental manipulation faced 
the research team in a study on changing attitudes in a housing 
project (14) A previous survey shoued that members of the project 
had hostile attitudes totvard one another, were ashamed of living 
in the project, and had little contact with either their neighbors 
or the surrounding townspeople The experimenter wished to test 
the theory that stimulating contacts among project residents under 
favorable conditions would reduce the hostile attitudes, which 
m turn, would reduce the shame at living in the project, which, 
in turn would cause the residents to initiate more social contacu 
with the town The people in the project, however, felt no need 
or increased contact In fact on the basis of past failures, they were 
pessimistic about community activities Furthermore, the expert 
organizer on the research team had no clearly estab* 
^ ^ projcct It was clear that she did not belong to the 
research tearn"^ related to the university 

which with a meeting of residents m 

o oL rf ’ "“rsery school project, a recrea 
recreanon acfi T children and adult education and 

munitv wide in * anning committees svere formed, and a com 
resource exueri!^!""® further discussion of plans with 

during the first ^ P'‘c&cess rvas made in organizing these activities 

hshed leaders They apparemr'™'’''’' 

support to a possibly comn "cw activities as giving 

rumor branding the'activme leaders Eventually, a hostile 
of all community activities t* *^"'"''*'>‘stic forced the suspension 
combated by the nrese weehs until the rumor could be 

sponsorship and pLpomT nF^th'’^ detailed information about the 
efforts to integrate the old lean P™STam, by deliberate 

Statements by the local n activities and by 

had been demonstrated^t°^^^ manager to the effect that the rumor 
other community activitieTwer “"'°“"dcd Subsequently these and 
a total period of eight months' “P continued over 

mental procedure can be f detailed record of this expen 

illustrates two errors which w^eT PP This experiment 

could have been avoided m ih “"^'^“'^'cted but which probably 
e first place if foresight had been 
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as good as hindsight First, the role of the community organizer 
should have been structured more clearly for the residents secondly, 
the hostility of the community leaders should have been avoided by 
not appearing to support the competing leaders 

Cooperative Relations 

We have already indicated (see p 108) that cooperative rela 
tions must start in the planning phases of the project in terms of 
selecting a setting, a problem and a sponsoring organization in 
'vhich the possibilities for mutual benefits from the research are 
maximal Lippitt (31) summarizes the reasons for successful coopera 
tion on a training workshop in intercultural relations as follows 

It seems probable tn retrospect that this collaboration of 
social practitioners and soaal scientists was successfully initiated 
and carried through to fruition because 
The social practitioners— 

—felt keenly the inadequacy of present accomplishments in 
community improNement of intergroup relations 

—were deeply sincere in their desires to achieve results rather 
than just to receive recognition for noticeable efforts 

—believed in the potentialities for constructive change m the 
intergroup relations of conflicting groups in the communit> 

—had a hunch that the methods for bringing about these 
changes might be more elficiently discos ered by the application 
of scientific methodology than by the usual trial and error efforts 
of experience unguided by systematic fact finding 

—were ready to liave faith tint this parucular group of social 
scientists v^as sensitive to the requirements of an action job (i e 
running a vsorkshop) is well as being adequate research tech 
nicians 

—perceived that these social scientists had strong personal 
motivations to do something about mtergroup relations in addi 
tion to a need for more scicnnfic understanding 
Tlie social scientists— 

—had a major interest in the scientific study of social change 
—had arrived at the h\|>othcsi$ that fruitful research on social 
change demanded an integritecl scnicc and study relationship to 
groups outside the ualls of ihe laboratnrv 
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—had as a pan of iheir research team personnel skilled in 
group action techniques as avell as measurement methods 

—had arrived at the conclusion that experimental method 
ology could be successfully applied to the study of the life of 
small and large groups in the democratic community 

—had strong feelings that inicrcultural and interracial con 
flict besides being a challenging scientific problem was a 
priority social issue 


This description fits Dcmsch s theory of cooperation (11) 
^ cooperative situation means creating a condition 
sue at if one party achieves his purpose or goal, the other party 
a t e same time achieves or is brought closer to his goal Conversely. 
1 is important to avoid a competitive situation, in which locomotion 
his goal will move the other party further 
tiv '^8°^ ^cutschs research shows that, given a basic coopera 
V ^ competitive situation, one can expect mutual 
imnrnv^w”’ understanding of one another s communication 

mnnir and other forms of behavior com 

monly considered cooperative (12) 

complex the experimental treatment, the 
tive relation!^ ^ terms of mamtaming the coopera 

motivation*:. Ifi, ^ ^ ^®’*5tant awareness of ones own goals and 
the experiment related to those of others involved m 

m tryine to car ^ resistances encountered 

study Stemm d experimental procedures in the housing 

coop“:Lno:;''' .here was no 

the perception of a ' members of the housing project As 

residents, these resistanTe*^'^"" in more and more 

establishing a cooperat.vVrTato'V'"”"' 
which has to extend to more 00 ^ '’ ^ ^ P™”'* 

during the course of the fielH ^ otsamration 

implied for the experimenter Tsvo prescriptions are 

cemed just what you ate goinv I? T 
(2) provide enough time Lou^ h ” 

non so that these peoole ran if and enough commumca 

in the sense that they'believe thaTlT"’', 

he will do nothing to harm them 
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or to conflict with their interests and, indeed, that he has some post 
tive concern for their welfare 

The social scientist may be able to rely on the skill of an experi 
enced practitioner, and cooperation with him will sometimes lea 
to the solution of the difficult problem of carrying out the experi 
mental manipulation This solution was successful m t e 
ment on participation because it was possible to o ^ ^ 

gent cooperation of the production manager, who had the necessary 
skills to conduct the meetings successfully and a position o aut or 
ity in the company Many of the problems which have been a“^cked 
through field experiments have, in fact, been pro ems ^ ^ 
rather highly developed professional skills in sue areas as uma 
relations training, therapy, community organization, etc or sue 
problems there are already available highly skilled professional 
people who may be called upon to conduct the manipu ation 
housing study is a case m point where a professiona commum y 
organizer svas employed as part of the research team Ezriel points 
out that even so complex a skill of the psychoanalyst as the making 
of an interpretation in terms of unconscious impulses fantasy 
determined reasons can be used to conduct rep ica e e p 
■nents within the psychoanalytical session (13) 

Executing the Experimental Treatment 

Once the basic role relationships are storked out, there should 
fie a collaborative diagnosis of the situation by I e resear 
at least some part of the client organization The purpose 
diagnosis is to assess the various factors that ssill be " 

executing the research dcsign-tlie resistances that may e 
i«ca, the d 5 nam.es of the situation in regard 
bringing about a change, etc The ssays of going a ou su 
laboratise diagnosis ssill vary tremendousi), depending ”P 
problem to be studied, the setting etc In the example o p p 
bon, .1 s.as possible for all those mvohed m the j']; 

on both an exlcnsise practical experience in t w t 
mns.derablc amount of relcsant research data 

lelcsanl experience and information slioiild be usrf _ ,hMrcti 

'be source, a useful diagnosis will have to be formulated m theoret. 



126 Research Settings 


cal terms Because it is necessary to predict reactions and to bring 
about planned changes, the diagnosis must go beyond the mere 
statement of facts or the labeling of things as ‘ good ’ or ‘ bad”, it 
must move toward more causal thinking, for it is only by manipu 
latmg Its causes that we can vary our dependent variable Getting 
e p roin the practitioner is both a way for the scientist to increase 
IS own lagnostic sensitivity and a method for maintaining the 
invdvemenf and interest of the practitioner (22) 

Hi-> ^ Step should be joint planning, based on an adequate 
the^inX^' ^ actions that must be taken m order to manipulate 
factor-; variable and to control other possibly confounding 

bevond thp ^ ™ost effective, this collaborative planning must go 
what ren ^ explaining to the client organization 

In mlk2 c J-nt deci 

design is to n>ethods and techniques by which the 

enough involved i?^' ** desirable that the practitioners be 

hind it ^ some of the reasons lying be 

some parts of th2cr^**^ nieans that the experimenter has to tram 
practitioners for examD'le'"^^n T''°" 

method before thev about scientific 

o£ having a control 

of management m r> ^ "®fd experiment A common reaction 
■stowanftoempWttT. '"“k- promising 

be necessary to continue employees, whereas it will 

in order to evaluate th? employees on the old basis 

>s quite important to take^meT" ” 1 ! procedures Heie 

the control groups are ne practitioner to learn why 

understanding can contnbur^*^ increase in scientific 

eral experiments, including- ^ practical objectives Sev 

tion, indicate that the ^*ample on the effects of participa 

can be involved in the plan2*^^”* client organization who 

support in carrying out th«. more widespread will be the 

In field expert 

of collaborative planning special problem 

faces the experimenter-namelv th organization frequently 

to carry out certain experiment ^ of secrecy In order 

» It may be necessary, at least for a 
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certain period o£ time, that all the subjects in the experiment 
possibly all of the people in the client organization, be prevented 
from learning fully the purpose of the experiment or some ° 

the procedure. Such secrecy is necessary if knowledge ^ *e ^ 
potheses would confound the results of the experiment. c ’ 
it is possible and desirable to give a full explanation at con^ 
sion of the experiment, but even where this is one e P 
secrecy can be a serious disturbance to good re ations. n 
Study, for example, the experimenter was testing t e ypo 
improved interpersonal relationships within t e P 

stimulated by the community activities would resu t in sp 
initiation of communication between the housing 
surrounding town (14). Accordingly, it was essentia ^ 
search team, while stimulating community activities 
project, should not also stimulate activities between e p j 
the townspeople. But it was also essential for adequate “nmol that 
the subjects in the project not know about this reason £°^e e^ 
ment for fear they might initiate 

simply to please the experimenter. It this need for emp ^ . 
is faced frankly in the early phases of planning for , ^ 
and if it becoLs a part of the training of the client n research 
methods, then it need not be a factor disturbing the cooperative 

"'"“n"pfoLing techniques for the manipulation of varmbles suo 

cesstul sLtegy Ld tactics tor changing the variable ^ “ 

on a correct and adequate theory of change. For »amp hi. m the 

experiment on the relations between the P j hnstilitv of 

surrounding town, the manipulation of the degree o 
interpersonal relations within the project was °n ‘h”pea;;; 
diagnosis that hostile interpersonal relations exi p Y 

because of autistic hostility and on the specific ‘bn 

Hostile relations could be changed by *'™“'=“'nE cornmummjmn 
among members through any successful group ac i\i ' 
if these theories and the assumptiom upon "b'* ^ 

"ere incorrect, then the actual manipulation > hostility 

tics in the project would not in fact cl.angc die v-f e of homl^ 
cl imcrpcrsonal relations. In the long run, ‘byefore. impros emen 
in our theory ot change will increase our abihty to plan succcsst.il 
^manipulations. 
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Lewin (30, p 193) has pointed out the importance of dealing 
with the total situation when manipulating a variable 

To vary a social phenomenon experimentally the expen 
menter has to take hold of all essential factors even if he is not 
yet able to analyze them satisfactorily A major omission or mis 
judgment on this point makes the experiment fail In social 
esearch the experimenter has to take into consideration such 
factors as the personality of individual members the group struc 
ture ideology and cultural values and economic factors Group 
p rimentation is a form of social management To be successful 
hke social management has to take into account all of the 
anous actors that happen to be important for the case in hand 
g^’oups Will therefore lead to a natural 
sciem^JT ° w-ll force the social 

mine grouVlTfe*""' '''' "'>'ich deter 

theoretical hyp™h«es 'I't^’u muln “‘'med to the testing of 
tittoners The"maniDiiIaf ” 1 *!'^ difScult to use skilled prac 
no. been done bXror rj" ' be one that has 

experience thus it penetieal 


experience thus it iv.ti "O any large body of practica! 

■nentertoco„d„,t J be necessary for the ertper. 

ments. »*. /^^nipulation himself In these field expen 

investigation is actually some 


theoretical hvoothp^n under investigation is actually some 
ulation be valid m ibp '^^P^ff^nt that the experimental manip* 
sabd-i e. It should conform' “ nneasurement should be 
m the hypotheses conceptual definitions involved 

This IS an additional diHin i 

nianipulation because it so trr *" out the experimental 

ntenter If, for example ^ ^’educes the leeway for the expen 

JVhat are the ellects'^rroTe 2“'' a question as 

group discussion? the exncrt^ as compared to the effects of 
conducts these it>o manipular!^«”*^*^ freedom in how he 

'■anous meanings In contrast n broad and 

plxying studied the more experiment on role 

llonal and altitudinal clianaei ypnthesis that ' deeper emo 
IS deeply invohed m the process or'’'^T*““‘* 'be subject 

It IS important that the exnrr. Plxying the role In this case. 

cxpertmenul man.pula.ton actually con 
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form to the conceptual definition of degree of involvement 
role rather than to some other similar independent variable The 
methods used for creating three degrees of involvement were (1) t e 
subiects actually partiapated m playing the key role ^ 
drama, (2) the subjects were instructed to identify with somebo y 
else whom they observed playing the key role, (3) the su jects were 
instructed simply to observe what was going on in the group where 
the sociodrama was presented Obviously, this procedure does va^ 
the degree of involvement, but this variable is not e ne ear y 
enough to permit accurate judgments as to how well t e manipu 
lations lit the definition. Probably other factors were vane 
taneously so that the experiment was to some extent con oun e 
For example, in one treatment, the attention of the subject was 
directed toward the key figure being observed, whereas m another 
treatment the attention of the subject was directed towar t e group 
as a whole One might say that direction of attention was systema i 
cally confounding these two degrees of involvement, thus reducing 
the validity of the manipulation 


^fohlems of Measurement 

Especially where the field experiment is designed to study 
theoretical hypothesis, it is essential for the researcher to measure 
the success of the manipulation used, for it is on y y re *1*5 
results to some known procedure that knowledge can be 
Where one is dealing with complex experimental proce 
measurements themsehes may be quite complex, as in tie cas 
the housing study Even an extremely condensed reporting of the 
mtpenmental manipulations used in this study require i 
pages In other cases, the manipulation has been measured q^nu 
tatively, as in the observational measurement o a ° 

Sions o[ the training process in a workshop (31, Chap ^ , 

anal) ttcal experiments, the measurement of the success o i 

nlation will L simpler In Rosenberg’s (38) experiment, the imp cs 

measurement was to ask the subjects in a 

<l>cy had Identified with the key figure as instrueled 

"le total group as instructed The answers to 

‘bowed that on the whole the instructions took. . 

"ere some exceptions When these latter subjects were rccatcgoritcd 
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m the analysis of the data, it was found that the hypotheses were 
even more clearly supported than when the subjects were analyzed 
in accordance with the treatment to which they were assigned Thus 
measuring the success of an experimental manipulation not only 
gives some indication of the degree to which it was successful but 
can also permit more valid manipulation through the elimination 
or reclassification of unsuccessful cases in subsequent analysis 

In a field experiment, most of the problems of social psychologi 
cal measurement are the same as they are for other types of research 
(The reader is referred to Chapters 6, II, and 12 of this book, which 
discuss measurement) However, a word of caution should be added 


concerning one point In most organizations, certain sorts of social 
psychological data already exist in the form of records used for 
administrative purposes In industry, for example, one frequently 
finds records of productivity, of absenteeism, of turnover, and of 
other aspects of human behavior It is tempting for the social 
psyc 0 ogist to use all these data which are available with no 
expen iiure of time, money, or effort on his part Yet it seems 
spread experience that such administrative records are 
Tinf delusion for research purposes First, since they are 

eyart purposes they do not always measure the 

to mflr/nL ® by one's hypotheses Thus they tend 

lonp run , from a theoretical orientation in the 

ofanadp* ? i! ^ serious hindrance to the development 

^“ch records are usually 

sonnel whn roiip/-» * be exact, and most of the per 

standards of accuracy Funh'^ trained in scientific 

moliiated and therefore o' frequently 

for example, resulted in a i^rl^ experimental procedure 
production which the researAeot 
of reduced tension (30, pp 217 2™ ^ 

ever, ue diimvprpd ti,-. ^ subsequent interview, how 

records only a part of her'prtSIamr ’'t ‘‘'l 

production was ,0 high thatX fe^er 
word of cannon is no, ,o say I" 

regard records collected for adl* '''“tircher should alsyays dis 
tninistrative purposes, rather, he 
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should avoid being overoptimistic about their use and should use 
them only when he has an adequate check on their accuracy 


Ethical Problems 

Field experiments involve new and more difficult problems 
of professional ethics and some more difficult than those in labora 

tory experiments or field studies in which no changes are in ro 

by the experimenter In a life setting, the experimental P™“dure 
IS, in fact, social action, sometimes in a situation o socia c ’ 
in which there are differences in values among t e peop e inv 
(17) In such a situation it is especially important ‘he expen 
menter to he guided by a code o£ professional ethics No generally 
accepted professional ethics, particularly for this ‘ ’ 

has yet been developed, although the American syc o ogi 
ciatton has been working on the formulation of sue “ mpnt 
mgly, this chapter will not attempt the difficult task, for the moment, 
each researcher must work out his own ethics 

It should be pointed out, however, that the e=tper™ental 
methods which have been recommended here are J! , 

tral Maximizing the mutual benefit of the research to both the 
practitioner and the researcher, collaborative ** 

tion in planning the research, open dealings wit research- 

tions, and eduiiting the participant to untostand he;<tsea‘“ 
all are specifications of a democratic ethics These me w 

■ng with people are based on an explicit recognition of the ultimate 
'alue of each person and l.is right of self-deiermination 


A SUMMARY OF THE ROLE 
OF FIELD experiments 


Tlie practical advantages of a field experiment 
tut and simple Anyone who wishes to take ‘ ‘ 

m any setting can improve upon Uie uncontrolled o' 

methods by the application of more scicnlific cxperim p 
dure, Thmugh iareful measurement, better 

ut control groups, and other aspects of improve.1 experimenial 
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design the practical problems of social action can be solved with 
greater certainty, with greater accuracy, and sometimes with greater 
efficiency than through common sense trial and error methods 

The primary scientific advantage of the field experiment is that 
It permits a more unequivocal determination of causal relations 
It permits determining cause and effect where a field study would 
reveal onl) a correlation Secondly, we have seen that the field ex 
periment is particularly appropriate for studying methods of social 
change, social processes and social influences Thirdly, the field 
experiment, since it deals with the total life situation, is well 
adapted for studying complex syndromes and social processes where 
the interrelationships among several more analytical variables are 

invniv^^H ‘ 


But the very fact that it deals with the total life situation also 
ea s to one of the major limitations of the field experiment— 
ameiy. it is not an appropriate method for studying with analytical 
as a Specific single hypotheses Finally, we must mention 

beraiKP method the difficulty m carrying it out 

settings whi h required and the contacts necessary with 

the skills atiH ^ research opportunity Even where 

necessari ?n experiments 

eSnt ' *hey may be m 


of research^in*"^^ scientific role for field experiments is in a program 

succers?;n;“rCfieTf 

and exact knLledgeoah^ett.n"'^'^'?'''^ ^ extensive 

to maniDulatp and thus enable the experimenter 

de^eIop^m :ftai?he^' 

mcnts which test the ranE^oTl!' ”1“*'" horizons for field expert 
at in the laboratorv IW™ ^PPhtation of generalizations arrived 
experiment attem^ to tcsr^r*"’!’ 

contribution to science hypotheses, it will make a 

tical value ’ °‘he™tse it will have a more limited prac 
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Laboratory Experiments 


Leon Festinger 


undentandmg or conTrS of 

world Neverfheless l,h ’ ^ o® >hoy occur m the real 

stgnificant part m the devH “P^’^^'oKon generally plays a 
have some understand of a saence It is important to 

function which lah \ ^ should be true and of the exact 

to the science as a whole'^ ®^Pc*''mentation should have m relation 

jory “PenmtnTauSirmd^Srit"^ ““ 

how the results of surh ^ laboratory experiment is and 

world • U woma be relatitSrr.™’ 

experimentation by mean r discuss the role of laboratory 

but ne shall attempt, rather" ,“‘“”*P’'^“ 'he physical sciences, 
examples from the problem* "*”**^*e the points to be made by 
doing this we may not be abirf* ">0‘“1 psychology Although by 
Otherwise be the case ^\e h ° clearly as would 

meaningful and carry more ^he discussion will be more 

the field which is now undei-"^-^« “ ** entirely oriented tossard 

unucr consideration 
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THE NATURE OF LABORATORY 
EXPERIMENTATION 

What Constitutes a Laboratory Experiment 
in Social Psychology? 

A laboratory experiment may be defined as one in which the 
investigator creates a situation with the exact conditions he wants 
to have and in which he controls some, and manipulates other, 
variables He is then able to observe and measure the effect of the 
manipulation of the independent variables on the dependent van 
ables in a situation in which the operation of other relevant factors 
IS held to a minimum Such a definition is, however, a great over 
simplification Given the techniques of experimentation today avail 
able, an investigator can at best achieve only a rough approximation 
of the degree of precision implied by the definition As better tech 
niques are developed, more control over laboratory experiments 
will, of course, be possible At present, however, we must include 
under the term “laboratory experiment’' a wide range of studies 
With varying degrees of control and precision 

We shall attempt, largely by means of examples, to distinguish 
between what might properly be called ‘ field experiments an 
' laboratory experiments ’’ In many cases, of course, the distinction 
IS clear and easy to make, in other cases it is difficult to maintain 
In general, we shall be guided by the two parts of our definition 
whether or not there was an attempt to create a specially suited 
situation, and the degree of precision in the control and manipula 
bon of variables 

It would seem clear that experiments in industry such as have 
^een described m the preceding chapter should not be called lab 
oratory experiments There is little or no attempt to set up 
conditions Typically, the situation is accepted as it is found and 
some manipulation is imposed The manipulation of the independ 
ont variable is usually a simultaneous manipulation of a set of 
factors The degree of control obtained in these experiments is 
Usually not sufficient to guarantee that the effects observed are 
unequivocally related to the manipulation of the independent 

'ariable 
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Let us compare such field experiments with the Lewin, Lippitt, 
and White study (21) on autocratic and democratic atmospheres 
This was a relatively early experiment in social psychology and 
IS perhaps close to the boundary between laboratory and field 
experiments In this study a number of boys’ clubs were set up 
for the express purpose of performing the experiment There was 
no real life situation which was taken as given Rather, a special 
set of circumstances was created because it was felt that the situ 
ation thus achieved would be an appropriate one for the study of 
the variables in which the experimenters were interested In this 
sense it should properly be called a laboratory experiment, although 
Its precision is perhaps not very much greater than the precision 
of an experiment in industry, such as the one reported by Coch 
and French (7) 


rtf . 1 , " ^ j White experiment, the manipulatu 

hftv* ^ variables consisted in having one leader of 

^ certain prescribed manner as compared tc 
twn ^ another club who behaved quite differently These 

the hpJi ° ® avior, which produced measurable differences in 

manv d2°n differed .. 

about all a aaperimenters were undoubtedly not clea 

lattna and f differences created Thus, rather than iso 
variable, manipulating a single variable or small set o 

ulation Ti, ^ ®^P®^™®ri^ers attempted a large and complex manip 

IrinTecSL^ir"” “P 

m maninni-jf ? control achieved and the degree of refinemen 

mdistinaunh'K? ‘"dependent variables, this study is probabl; 

WeTan o' fi''d experiments 

ment wuh a re^r„'°r'u'' “ example of a laboratory expert 

experiment by FestinLrnm pveasio". 

an attempt was voting behavior In this expenmenl 

not the subiect, taetor-namely, whether oi 

of the CTouD r robgious affiliation of the other memberi 

experiment express purpose of the 

group was imtiali^^^^ ^ahen to ensure that every member of the 
comparable conri ^ ^ ®^ranger to every other member Exactly 
for whom subierfl°"" nominees 

behavior was standarl° 4 always paid participants whose 

>zed These same paid participants identified 
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themseUes as having different religions in the different experimental 
groups, thus controlling for a wide variety of personality factors and 
first impressions 

In such an experiment we can be more certain than we can in 
a field experiment that the results obtained are due directl) to the 
\anable manipulated by the experimenter It is probable that a 
\'anable such as ‘ whether or not the subjects know the religious 
affiliation of the other members is still not a fine or precise factor 
It IS probably once more, a cluster of factors A laboratory expen 
ment should, howeser, attempt to refine the manipulations as much 
as the present state of knowledge permits One of the marks of 
progress in a science is the extent to which such laboratory manip 
ulation can be refined and specified 

There is frequently a tendency m social psychology to criticize 
laboratory experiments because of their ‘ artifiaahty A word must 
be said about this critiasm, because it probably stems from an 
inaccurate understanding of the purposes of a laboratory experiment 
A laboratory experiment need not, and should not, be an attempt 
to duplicate a real life situation If one wanted to study something 
m a real life situation, it would be rather foolish to go to the trouble 
of setting up a laboratory experiment duplicating the real life con 
diiion ^vhy not simply go directly to ilie real life situation and 
study u? The laboratory experiment should be an attempt to create 
a situation in which the operation of variables will be clearly seen 
under speaal identified and defined conditions It matters not 
'\hether such a situation would e\cr be encountered in real life 
^n most laboratory experiments such a situation would ceriainh 
UfX'er be encountered in real life In the laboratory, howe\er we 
find out exactly how a certain \ariable affects beha\ior or am 
tudes under special, or pure, conditions 

This is certainly not the end of the task One must also find out 
'low these sariables interact with other sanables The possibihts 
of application to a real life situation arises when one knows enoiigli 
about these relationships to be able to make predictions concerning 
a real bfe situation after measurement and diagnosis of the stale 
of affairs there 
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Let us compare such field experiments with the Lewin, Lippitl, 
and White study (21) on autocratic and democratic atmospheres 
This was a relativel) early experiment m social psychology and 
IS perhaps close to the boundary between laboratory and field 
experiments In this study a number of boys clubs were set up 
for the express purpose of performing the experiment There was 
no real life situation which was taken as given Rather, a special 
set of circumstances was created because it was felt that the situ 
ation thus achieved would be an appropriate one for the study of 
the variables in which the experimenters were interested In this 
sense it should properly be called a laboratory experiment, although 
Its precision is perhaps not very much greater than the precision 
of an experiment in industry, such as the one reported by Coch 
and French (7) ^ 


»i, j t-ippitt, and White experiment, the manipulation 

. ® variables consisted in having one leader of a 

annth^'i ^ Certain presCTibed manner as compared to 

iwn f behaved quite differently These 

thp ^ ® which produced measurable differences m 

manv members, were complex and differed m 

about an experimenters were undoubtedly not clear 

latino- differences created Thus, rather than iso 

variables ^ manipulating a single variable or small set of 

ulation ^’^perimenters attempted a large and complex mamp 

cl^t lnTe^nf ^ ‘■"'t' ""-P' - “P 

m maniniilat ^ r ? control achieved and the degree of refinement 
indistiiiLishlbW variables, this study is probably 

ment with a re°riiX’'h“’i.“a =* laboratory expert 

experiment by FesttucJrnm P’'“*‘°"’ 

an attempt was voting behavior In this experiment 

not the subjects w.!. ^ ® single factor-namely, whether or 

of the grouD r religious affiliation of the other members 

experiment. with°ra^* express purpose of the 

group was muiall ^ ensure that every member of the 

comparable conn,»7 ^ s^r^nger to every other member Exactly 
for vshom subierr,°"^ created for each group The nominees 
bchaiiorwasstandarr"? ‘‘‘"“I'' P""^ participants svhw 

•red These same paid participants identified 
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effect of friendship existed in the absence of differential amounts 
of contact It would enable one to accept or reject the third hyPO* 
esis stated above In other groups one could experimenta V ™ 
the accessibility of other members for communication 
evidence as to whether the friendship represented a decrement i 
restraint against communication or whether there were ac ua p 
sures to communicate in the specific direction of rien s 

Such an experiment would undoubtedly be "P- 

but. since the major body of this chapter will devoted to th 
discussion of how to perform such experiments an ow p 
the desired conditions, we shall not. at the moment, go into the 
details of how it might be done Let it sufBce now to say 
the laboratory, by setting up an artificial situation, *ould be 
able to verify, elaborate, and refine our know e ge so a ],„„j(i 
our understanding of important processes in socia i 
be stressed again, however, that the problem of -PP>‘““°" 
results of such laboratory experiments to the rea i e ® 
not solved by a simple extension of the resu t u 
requires additional experimentation and study is , 

important that the results of laboratory exjierimen ^ runnina 
m real life situations Unless this is done the dnng^' ® 

dry’ or hitting a dead end ’ is always present A 
play between laboratory experiments an smdies o 
ations should provide proper perspective, for the res 
should continually supply new hypotheses for ® 

retical structure and should represent progress m the 
the problems of application and generalization 


^ifficulttes of Performing Laboratory Experiments 

Laboratory experiments, however, do not 'j 

road to the "ollecuon of data for the resolution 
problems In social psychology the, are typimlly > ' 

and many dangers are present in their execu lo results 

dilhcult m creaTe in the laboratory forces strong 
to be measurable In the most excellently one a prodnecil 

ment, the strength to which diflerent "an 

>' extremely \vcak compared lo the sirengt i obtain 

ablcs exist and operate in real life situations One is able to oblain 
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The Relationship Between Laboratory Experimentation 
and the Study of Real life Situations 


In the conducting o£ research, there should be an active interre 
lation between laboratory experimentation and the study of real life 
situations It is relatively rare in social psychology that hypotheses, 
hunches, and recognition of important variables emerge initially 
from the laboratory, most often they arise in either the formal or 
the informal study of real life situations In studying real life situ 
ations, we are forced to deal with the factors and variables as they 
exist in all their complexity Because of this complexity and lack 
of control, it is rather rare that definitive conclusions and une 
quisocal interpretations are reached m such studies, but frequently 
new variables and new hypotheses are brought to our attention 
One can take these suggestions, hypotheses, and hunches and u.c 
a oratory experimentation to verify, elaborate, and make more 
s^re the theoretical basis for the empirical results which have been 
obtained 


In the laboratory experiment, sufficient control can be achieted 
o 0 tain efiniiive answers, and systematic variation of different 
ac ors is possible As a result of this greater control, precision, and 
manipulabihty conclusive answers can be obtained and relatively 
theoretical points can be tested For example* 
fnimrl ^ of a rumor in a community (11), it was 

in hivp more likely they weie 

lint ^ e rumor This finding may suggest the hypothesis 

ivnes of rnm*^ reduces restraints against communication of various 
of a frifnri *oggest the hypothesis that the existence 

Iv Zlty. to communicate, or .t 

moL more friends see 

quentlvaJl spend more time with these people and come 

In a laboni opportunity to hear the rumor 

m w nch ^^Penment u would be possible to set up a situation 
uo^ld taTe ™ ^ ^‘Sor. cCeerdata winch 

One could, lor examni ''r'”' interpretation, 

together m ,., 1 , u groups of strangers and friends mixed 

opportumtv f amount of contact among members and the 

held among them were experimentally 

e results would enable one to say whether the 
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effect of friendship existed in the absence of differential amount 

of contact It ^vould enable one to accept or reject t e t ir 

esis stated above In other groups one could experimenta y va 
the accessibility of other members for communication to ob am 
evidence as to whether the friendship represente a ecremen 
restraint against communication or whether there were ac ua p 
sures to communicate in the specific direction of rien 

Such an experiment would undoubtedly be i cu t o se 
but, since the major body of this chapter will e evo e 
discussion of how to perform such experiments an ow o p 
the desired conditions, we shall not, at the moment, go ' 
details of how it might be done Let it suffice now to 
the laboratory, by setting up an artificial situation, we s 
able to verify, elaborate, and refine our knowledge so as 
our understanding of important processes m socia i e 
be stressed again, however, that the problem of 
results of such laboratory experiments to the rea i e 
not solved by a simple extension of the result Sue PP ^ 
requires additional experimentation and study Js , 

important that the results of laboratory experiments ^ 
m reaUife situations Unless this is done the danger o g 

dry- or hitting a dead end ' is always present A 
play between laboratory experiments and studies o . 

aliens should provide proper perspective, for the 
should continually supply new hypotheses for ui S 
retical structure and should represent progress in the 
the problems of application and generalization 

difficulties of Performing Laboratory Experiments 

Laboratory experiments, however do not 
toad to the collection of data for the reso ° , jo 

problems In social psychology .hey are typi^lly ' 

and many dangers are present in their execu lo results 

difficult to create in the laboratory forces strong expen 

•O be measurable In the most excellently done 
ment, the strength to whtch d.fferent ™n:.bles can be pmduerf 
extremely ueak compared to the strength s«t t obtain 

ttbles exist and operate tn real I.fe s.tuat.ons One ts able to obtain 
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results and to see clear!) how these variables operate, m spite ol 
this weakness because of the increased control one has in the 
aborator) situation But it is always possible, even probable, that 
le actors wi I be so weak that no differences between conditions 
created are apparent in spite of the increased con 
\ ^ setting up of a laboratory experiment, especial 
can variables as strong as one possibly 

succeedpH ^ f ^ determine whether or not one has 
cenerali 7 tt!^" ^ k experiment is over An exception to this 

be seen m a? h weakness of laboratory manipulation can 

hers f2i Th announced perceptions of group mem 

^ 

tory situation^K^^h*"^^^!^ strength of forces in the labora 

simultaneously In the clm^nl'^^ of manipulating several variables 
here concerned n r ^ research with which we arc 

effect theoretically important to see the 

Unfortunately, howe\er^'!li °f two or more variables 

‘empts to manipulate variables the experimenter at 

'inable This is esnpr.^n fo"'er will be the strength of each 
>s to be done bv m^n ^ ^he manipulation of the variable 

of “> The 

njcni, that the number of v*... technical develop 

late simultaneouslv m th *^hles which it is possible to manipu 
"ill undoubtedly become ^ o*^*ot7 is relatively restricted This 
manipulating variablps more powerful techniques of 

, Th«e d,mcul,.es h^ve t 

elusions one can draw fm u ’’’Pocunt implication for the con 
As m any study, u is nossiW *.i^ ‘‘^^ults of laboratory experiments 
entirely irrelevant \arwbles-ih'^* experimenter is dealing svitli 
tionship among the vamKw *here may actually be no rela 
dition would result m hemg studied Such a con 

eiween experimental *‘es«lts— that is no differences 

also find a lack of difrerenr!?T”* However, we should 

conditions if our experimental experimental and control 

strong to reveal measurable difT sufficiently 

really exist Thus, negative though such differences 

can mean very httlc indeed If** ^ ^ laboratory experiment 

We obtain positive results— that is, 
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demonstrably significant differences among conditions— we can be 
relatively certain concerning our interpretation an cone usion 
from the experiment If, however, no differences emerge, we can 
generally reacli no definitive conclusion unless we are quite certain 
that the manipulation of variables m the experiment 
successfully and adequately At the present stage o tec nica ev 

opment, it is seldom that we can be certain, in t e ^ ° 

positive results, that our manipulations were adequate 
edly, as more and more experiments are done, good eyi ence wi 
become available for believing that a certain manipu anon is 
adequate one, and then negative results can be mterprete as 
demonstrating no relationship At the present time, ^ 

too easy to set up a laboratory experiment which ecause o 
effective manipulation of \ariables, will show no i erences am 
conditions It should be stressed again that at the present ® 
technical development, negative results perhaps J pvnen 

that the experiment was not set up carefully ^d t at ^ 

menters attempted manipulation of the variables was i 
Keeping m mind these difficulties and the 
must exist between laboratory and field investigation evnen 

proceed to a more detailed examination of how a or y P 
mems can be performed 


THE DESIGN OF LABORATORY 
EXPERIMENTS 


The first and foremost requirement for a successful 

experiment is that the problem be stated , 

This means essentially that there must be a ig e^ 
fiaty and danty in the statement of the problem an i 
nuion ot the variables involved The foregoing implies that before 
one can successfully do a laboratory experiment one nlrea y 

know quite a bit about the phenomena one is ® , 

The process of specifying and clarifying the statemem °t a 
problem so that it is amenable to experimental ‘reatme^- W 
no means a simple or easy one Let us take an examp 
the kinds of pLblems which confront the 
t'age In a field study of transmission of a rumor ga 
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tion (4), It was observed that communication tended to be directed 
upward in the organizational hierarchy This result was explained 
as epending upon forces acting on members to move upward in 
t e organization, i e , the upward communication represented sub 
stitute movement on the part of the members 
,, , ^ set out to perform a laboratory experiment to test 

thoroughly At this point the statement of his 
lent i"' ^ What direction does communication 

IS still mnrK !” ^ Structured hierarchy? ’ This statement, however, 
ment An -.»» S^^neral and vague for the purposes of an expert 

makes u .mmXtdy derr\tat or' “ experiment 

Whit answer questions such as 

mun.r,!r rl" " "E-ctly what kinds □£ com 

IS customarily thoVght of 

lesels in ^ hierarchical structure Do superior 

>t so, whafkinds 0 ^’’ subordinate levels and 

hterarchy characterofn'ik'^^ ** successive level upward in the 
increased freedom of cho! attractiveness of the svork, or 

portance of the svorkf For^r ‘"“aased tin 

experiment thdarhA,.. r the purpose of setting up a laboratory 
be made more specifij hierarchy must 

laboratory on the t J establish a hierarchy in the 

the subjects holdmcr perceived importance of the job to 

exact ssirk iiaf ^^j*! a«n.cttvcncss of the job and the 

cation iNouId be cxocctpH 9^cstion of what kind of communi 
clear tint ^ distinction “P'’’'ard in such a hierarchy It was 

communication communi^ » made between work oriented 

information, and com~.“" “5 commun.cat.on of 

It i\as largely m the last eai "^1^'ch was irrelevant to the task 
the effect of substitute mn communication content that 

Consequent!), the cxperimcnT”^”* "ould be expected to appear 
encourage communication nt allow and in fact to 

in Kelleys c>:periment content The final problem 

irrelevant communication the direction of 

ceivcd differential importance *^of 11* ‘“^'‘^^^hy based upon per 

specific enough to permit the dcs.»J *^ task? This statement was 

Sn of the actual experiment Tins 
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process of clarifying the objectives of the experiment takes consid 
erable time, although it may not take long to describe after it has 
once been done 

The difficulties of designing a laboratory experiment are by 
no means overcome when the problem has been specifically defined 
There remain the major tasks of inventing measurement devices 
and techniques for manipulation of variables which wil c ear y 
measure and manipulate the variables which have been defined in 
the statement of the problem No matter how specifically and clearly 
the concepts are defined in the statement of the problem, the 
laboratory experiment cannot be successful unless the measurement 
and the manipulation of variables actually relate to these e ne 


concepts 

Thus, for example, in the Kelley (19) experiment mentioned 
above, it was necessary to develop techniques for pro ucing a 
hierarchy as defined, while other vanables, such as t ° 

done, power, and attractiveness, would be controlled T e situa ion 
created had to be one in which irrelevant communication would 
occur Adequate techniques for measuring the amount and direc 
lion of communication had to be developed In t e experime , 
a two level hierarchy was established Each level did exactly the 
same kind of work, although each was under the impression th 
the other level was doing something different Hig an ow 
archie perceptions were encouraged by the instructions ^ ^ 
jects one subgroup was told that its own job was the importa 
one, the other subgroup was told that the job of t e ot er e 
was the more important Communication of irrelevant materia 

was encouraged by having all communication came ^ 

and by injecting into the communication stream prepare cti 
*^otes which were irrelevant in their content, thus encouraging su 
jeets to do such writing themselves All notes were collected and 
^ept, and thus analysis of the content of the communication i 
direction, and amount was possible , 

It IS rarely safe to assume beforehand that , t 

*0 manipulate variables will be successful and wi tie i » , 
''»th the concept the experimenter has in mind t is a wo 
precaution to check on the success of the 

r‘ons In the experiment by Kellej. the subjects were asked a num 
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of questions after the session was o\er to determine whether or not 
the manipulation of status in the hierarchy had been successful 
It was found that, in terms of their reported perception of status 
and their desire to be m the other role, the manipulation had ere 
ated a difference between the two levels This difference was a 
relatively small one however Small differences in the results could 
e directly attributed to the small difference in perceived status 
When the difference in perceived status was made larger by selecting 
out those subjects for whom the experimental manipulations were 
c ear y successful, the results become much clearer and more coii 
c usive f there had been no check on the success of the expert 
menta manipulation, such analysis would have been impossible 
inrrt^Ai ^ impossible to attribute unequivocally the 

cxDpri results to the relative inadequacy of the 

experimental manipulation ^ 

»t ‘he manipulation of variables 
Sraio^ tv " ■" P”' P«'>n»n=ry studies In almost any 

Quacies design will have certain made 

prehminarv *^^come clear after a few trial experiments Such 
mve^^ .7, "r,! provide practice for the 

ardized bv instructions become stand 

ardized by the time the regular experiments start 


the execution of laboratory 
experiments 

ables can be or control of van 

laboratory expenmenr wt' staTat^’' “ 

'o coser in detail most of th,. 1 * '''= following pages, 

fruitfully and to ^ techniques which have been used 

y to give examples of their successful use 

ff— ehout S„h, CCS y„c«e Experiment 

Decisions about the kinds nf 

how they are to be reennf#./! j pei^ons to be used as subjects, 
before they come to the exo'eri"^ 

tiw for the manipulation of important opportuni 
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Controlling the Composition of the Group 

It IS possible to arrange the composition of the group so as to 
control the number of friends m each group or to select subjects to 
ensure that all of the members of a group are strangers to one 
another at the beginning of the experiment The decision concern 
mg the composition of the group depends of course upon t e 
purpose of the experiment and on the variables upon w ic t e 
experimenter desires to focus his investigation ^V^e shall give some 
examples of the introduction of an experimental control or manip 
ulation at this stage of the procedure 

The experiment by Festinger (10) previously re erre to a 
as Its objective the determination of whether knowle ge o re igious 
affiliation in a mixed Catholic Jewish group would a^fct the atti 
tudes of members toward one another It was assume t at ese 
attitudes svould be reHected by their votes m elections for officers 
of a club It ivas decided to have groups meet in the laboratwy and 
elect officers of a club into which they formed theinsehes Ha f 
the elections were to take place while no one in t e group riCTv 
the religious affiliation of any one else, the other a o . 
ttons were to take place after the religious affiliation of each member 
was publicly announced It was obviously essential, for th p 
cedure to be successful, that none of the six members of a y g p 
know one another Contact was made with nine colleges in the 
Boston area and permission to recruit 
"as obtained Experimental sessions were then sc e u e 
>n each group only one person from any one college was present 
Thus, when the group met, the six members eac i cam 
different college in the area and the chances of their know ing each 
other were quite low In spite of all these precautions howeier 
one out of 13 groups had to be 'Bminated because i ^ 

raembers did know each other, having gone to high sch S 

the other 12 groups, all the members were complete strangers 

one another i 

Schachter (26), in an experiment designed to 
telationships between difference of opinion am r j 
warned his groups composed of strangers lo mimmite .he effects of 
past history su* as li.abl.shed preferences or aiers.ons, ar^ong 
toembers Hating strangers was miporiant because he was panic 
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of questions after the session was over to determine whether or not 
the manipulation of status in the hierarchy had been successful 
It was found that, in terms of their reported perception of status 
and their desire to be in the other role, the manipulation had ere 
ated a difference between the two levels This difference was a 
relatively small one, however Small differences in the results could 
be directly attributed to tlie small difference in perceived status 
When the difference in perceived status was made larger by selecting 
out those subjects for whom the experimental manipulations were 
c early successful, the results become much clearer and more con 
c usive If there had been no check on the success of the expert 
mental manipulation, such analysis would have been impossible 
ivou a so have been impossible to attribute unequivocally the 
inconclusiveness m the results to the relative inadequacy of the 
experimental manipulation 

adequacy of the manipulation of variables 
labLamrv^^ preliminary studies In almost any 

quacie design will have certain made 

preliminarv^ l>ccome dear after a few trial experiments Such 
mvestieat^ rniportant to provide practice for the 

ardized hv th° ^1! instructions become stand 

ardized by the time the regular experiments start 


the execution of laboratory 
experiments 

ablcs can be or control of van 

>abomro;t;™trw:, hT' ^ 

to cover in detail i"*' attempt, m the following pages, 

fruitfully and tn techniques which have been used 

"y and to give examples of then successful use 

/nr the Expenmen, 

how they are to be^r Persons to be used as subjects, 

before they come to^th^*** ’ expect 

ties for the manipulatio'n'?”'"'™' Pt°'’'de important opportuni 
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It may be more difficult to manipulate vaiiables adequately On 
the other hand, if the experimenter plans on more than one meeting 
per group, he must expect that a certain percentage of subjects ’vm 
not return after the first meeting 

Designs which require the group to meet several times encounter 
another difficulty Many uncontrolled factors may be introduced 
since the subjects may contact one another outside the experiment 
and, m this way, materially change the situation between experi 
mental meetings The decision as to which of these two types o 
experimental designs to employ depends, again, upon the objectives 
of the experiment and on how these objectives can best be accom 
phshed A number of examples of each kind of experiment wi e 
given to illustrate the advantages and difficulties 

Deutsch (9). in his study of the effects of competitive and coop 
erative situations on group problem solving, felt that the full effec s 
of the experimental variables would reveal themselves only it me 
group would have considerable experience working together under 
the prescribed conditions He decided on six successive meetings 
of each group and, to accomplish this, persuaded the instructor of 
a course to give students credit lor participating in his experiment 
Under these conditions most subjects attended all six 
an arrangement is not usually possible, but it is genera y nec ^ 
>0 have some means of ensuring that subjects will return when the 
group IS to meet several times 

Schachters (26) experiment on rejection of deviates used 
meeting of each group It was necessary, however, for the subjec 
10 be under the impression tint they were to continue to meet once 
a week for a consiilerable period of time The experimenter re 
cruited subjects by idling them about clubs that were being o™ 
and giving them ihc opportunity to join one of the c u s u J 
"ere told tint by joining they were committing themselves to a lend 
die first meeting^ After the first meeting they would be able 
decide for themselves whether or not they wanted to continue 
In an experiment on strength of attraction to groups, i “ 
>'wd the number of meetings which subjects 

major measures of the slrenglh of their attraction to <he grot p 
Ue, too, gave subjects an opportunity to volunteer 1° , 

"htrh were to continue to meet every week Subjects c 
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ularly concerned with the effect of the experimental condition 
upon acceptance and rejection He recruited volunteers from courses 
which were divided into small recitation sections By scheduling, 
in any one group, only one person from any one recitation section, 
he was fairly successful m eliminating prior acquaintanceship 
In both the examples above, having strangers compose the 
group was a technique used to exercise additional control over 
t e experimental situation In experiments on the effects of dis 
cussion on opinions about matters of fact, Jenness (18) controlled 
t e range of difference of opinion m the group by the assignment 
given groups on the basis of their original estimates 
"IL , cT question French (16). m an experiment on the 
c s o rustration and fear, used the composition of the group 
a variable He was concerned with the 
eani 7 PH ^ of frustration and fear upon organized and unor 
cruitedafnl^ ^^n*^** unorganized groups he used subjects re 
first Mmp in h who met together as a group for the 

melbe^ h groups he used club 

m activities ^ ^ fustory of working together and engaging 
to ^mzed group came 

a cross onp ^ gather This type of manipulation is, of course, 

an unorgani 2 erone°Thr“‘’ f™® 

position ot a '''P' manipulation of the com 

m Foduce fine 

earliest expenment! wTth ' conditions Some of the 

major variable the presence®^'’'‘r' employed as their 

the person worked alone “r”^ ' '*> 

ence was found affect hi, perfo^aLf (8)'°'”'' " “ 

Dnrauonath, Groups E.,ste„c. 

experiment will be cond'*'*'^"’ ” ** "ecessary to decide whether the 
Will be required to c meeting or whether the group 

procedures has advantaTel*”^^!! sessions Each of these 

to be performed m onl ” “‘sadvantages If the experiment is 

volunteer. If the ex "? ”“‘■"8. ttts generally easier to obtain 

experimenter restricted to one session, however, 
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joining that specific club (low attraction to the group) ^ This manip 
ulation of attraction to the group was also shown to be success u 
by the results and by answers which subjects made to questionnaires 
after the experiment 


Size of the Experimental Groups 

No matter what techniques the experimenter employs th^e ill 
always be some subjects who, after having agreed to be at t e a 
oratory at a certain time, will not appear They may have forgoUen, 
they may have changed their minds or something may have ap 
pened which made it impossible for them to attend In any 
the problem for the experimenter is the same In designing a a 
oratory experiment in which human subjects are to be » **■ 
well either to design the experiment so that it may be conducte 
with a variable number of subjects or to make some provision to 
ensure the proper number of persons in each group It is gei^ra y 
most desirable to allow for variation m the number of subjects 
Thus for example, an experiment may be designed so that it can 
be conducted with either five, six. or seven members in the g-oup 

If seven persons are then scheduled for each meeting, an i su ci 

precautions are taken, ^ very few groups will be lost 

When a design requires a constant number of su jects in e c 
group, there are a number of techniques to ensure the Presence o 
the proper number Festmger (10). in his experiment on the ellects 
of knowledge of religious affiliation, felt it necessary to keep tne 
size of the groups exactly constant at six subjects per group 
of these were to be Jewish and three Catholic This essen la 
because of the desire to have the group evenly divided betvveen 
two religions Leeway in the number of subjects in eac group 
"ould have produced deviations from an even division which migh 
have introduced additional complexities Before each experiment 

I This IS not strictly an experimental ^ 

» reptesems sclcclion o£ subjects on the basis of some measure in 

Pwimpaie ihe loner rbe proporuon of subjects nbo appear nben scl.eum ( , 
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after the first meeting, whether they wanted to continue their 
membership Little pressure was applied to the subjects to return 
to subsequent meetings The number of meetings actually attended 
was assumed to reflect their attraction to the group 


Starting the Manipulation of a Variable 

It IS possible and sometimes necessary, to start the manipula 
t.on of an experimental variable at the time the subjects arc 
recruited for the experiment This can be done by providing van 
ous expectations for the subjects which will affect the attitudes with 
w 1 C t ey come to the experimental situation, or by collecting 
to manipulate a certain desired 
f ^ ^ examples of the experimental manip 

Sfvp f ^'■hich begins at the time of recruitment 

several experiments (3. H, 28) have varied attraction to the 
subiects ^ nianipulating the degree to which the 

memb r, ofib »e congenial with, the other 

“er”d o b ' ‘™' of who volun 

which conccrn^a^^c^^ ^skcd lo answer a number of questions 
they liked in ^'■^tteristics of themselves, characteristics which 
m other neonlc N characteristics which they disliked 

up" a'^Lp\t'r" P-'* ‘hese data in 

information ?hc ^ •I"' subjects had provided such 

groupTiCtl P'-^Wy able^o tell some 

and to tell others ib" 'n* another and be congenial 

of suchtprr™ m »= very congenial The results 

Schachter 1261 * * 'b'"' "'^'"pulations were successful 

banted to manmnl.,!” experiment on the rejection of deviates 

est in ihe activity ,n Xch'th" ‘"T 

subjects were asked to .n groop was to engage When the 

■n detail Those who was described 

on which they were information sheet 

were in joining each of iL rabngs of how interested they 

posed of subjeas who were ^rh “r’ 

club (high attraction to th/'*’’' *" ‘hat specific 

composed of persons wh k '^^ereas other groups were 

o nad indicated relatively low interest m 
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‘real for the subject and a situation which is experimental 
for him All of the situations are, in a sense, real for the 
subject, and all of them likewise, are experiments from the 
point of view of the investigator Some examples from other fields 
of investigation may illustrate our point more clearly If a psyc o 
ogist does an experiment in discrimination learning using rats as 
subjects, the situation is obviously an experimental one or t e 
investigator For the rat, however, it is undoubtedly a very rea 
situation The maze or discrimination box is a place where he works 
and gets led The basis of the ' reality of the experimental situation 
for the subject is somewhat less clear when humans are used as 
subjects Thus, for example in an experiment on level of aspiration 
the subject may come to the laboratory knowing he is to help in 
an experiment He is given a series of tasks to perform and is asked, 
before each task, what he is going to try to score on the subsequent 
task One may well ask, In what sense is this situation a real one tor 
the subject? Certainly it is not real m the sense that it is a situ 
ation similar to those which the subject encounters in the ordinary 
course of events, on the other hand, it is certainly real in t e sense 
that powerful motives are brought into play and strong orces are 
set up which act on the subject and determine his behavior i 
lawful ways Thus the situation in which one places the subject 
can be real for him in that it brings into play powerful forces 
regardless of whether or not it is cognmiely an experimenta situ 


ation for him , , » . 

If the situation is cognitively a real one for the subject it is 
probably easier to bring powerful forces into play t may e m 
dilhcult to produce equall) strong forces if the situation is cog 
nitively experimental In the latter case the strcngih ol ilie forces 
"Inch can be brought into play depends largely upon the relalioils 
hetween the subject and the experimenter, the motivations w iic 
made the subject decide to volunteer for the cxperiiiteiii, and 
desire to cooperate These forces can in the jiroper circumstances 
l>e quiic strong It is much easier to create a laboratmy s.tiiatiou 
''hicli IS cogniincl) expenmenni for ihc subjects o 
cogmtucl> ‘ rol s.iu-.i.on nnd sull be .blc to ""3 

“laic \ariablcs successfully maj require a great tea o , 

3nd much attention to technical details If the subject secs g 

die subterfuge, ilic uholc c\pcnmcm mas be iinaliilucd 
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each subject was tvntten a letter stressing the importance of her 
coming to the experiment On the day before the meeting, each 
subject was spoken to by telephone to make sure that she ivould 
be present In spite of these efforts, only five subjects appeared in 
a number of groups In most of these instances the subjects who 
had arrived agreed to wait while others who had volunteered were 
telephoned until an appropriate person was reached who agreed 
to come down immediately By this procedure very few groups had 
to be discarded In Pepitones (25) experiment on group produc 
tivity, the situation was designed so that it was essential to have 
t ree su jects present in each group The group was to work on a 
task which was divided into three pans, each of which had to be 
per orme y one subjea The experimenter scheduled four sub 
jec s or eac ^oup Occasionally only two subjects appeared and 
canceled most frequently three subjects 

suLtion 

exnprim^.'r* allowed to observe the 

experiment in progress 


the content and form of the 
experimental situation 


how the make a number of decisions concerning 

Sttnrn cognitively for the subjects, m 


or Expenmentar Sttuauons 

experimental to situation can vary from one which is frankiv 

by no means all dear P^^'b^ities svithin this range are 

types of experimental sttum.™“*™“ “ “"“■'"‘"S 

We shall, however, discus, superior for which purposes 

lead an experimenter to set uiTh considerations which might 
To discuss these groups in one or another manner 

somewhat farther the disadvantages we must explain 

the distinction between a situation which is 
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real aspects of the situation, it was also not possible for the experi 
menter tc engage in any further manipulation of variables while 
the meeting of the group was m progress These problems, in this 
experiment, were satisfactorily solved by the use of paid participants, 
a technique which will be described later 

In an experiment on the effects of knowledge of religious 
affiliation Festmger (10) decided to use a cognitively experimental 
situation This decision was made because it was obviously of 
importance to control the group session firmly and to carry on 
manipulations of variables while the session was in progress T e 
group consequently met with the knowledge that it was helping 
in an experiment They were told to imagine that they were a 
club There is no doubt that the forces in this situation were weaker 
than the forces which would have operated had the subjects actually 
been members of a club engaged in the same procedure By virtue 
of the cognitively experimental aspects of the situation, however, 
this disadvantage of weaker motivation was counterbalanced by the 
precision of measurement and the control of extraneous variables 


T/ie Choice of Activity for the Group 

The choice of the activity m which the group once assembled 
■n the laboratory, is to engage is somewhat dependent upon the 
tlecision concerning the cogniti\c rciliiy of ibc experiment icrc 
•s. of course, much leeway in the choice of actiMiy, althoug i it must 
be one which is consistent with the purposes of the experiments 
and does not conflict with the other expcrimennl decisions wlucii 
ha\c been made If the experimental situation is to be cogniiuc) 
real, there are restrictions on the t>pc of actiMiy '^hich can be 
employed If the sitintion is to be cogniii\cl> experimental, there 
>snuich less hmiiaiion and the selection of an actiMt) which is wen 
suited to the exiicrimcntnl purposes is easier The acnui> 
chosen to allow for the manipuHuon of the \irnblcs. t ic co 
of the measures m whicli the inscstigator is intcrcsict tmc le 
arousal of siiflicicntb strong forces so tint the effects sm »e m 
“table It IS imiiossiblc. of course, to Im all of the s-nnoiis aciivi ic 
‘n "hidi labontorv groups mn> engage Uc shall present a ew 
’samples of d.fTcrent kinds of aciixilies winch base l>een used and 
*he reasons for their use 
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We have then these relative advantages and disadvantages 
which the experimenter must consider when deciding whether to 
make the experimental situation cognitively experimental or cog 
nitively real for the subject If the experiment is cognitively real 
It will be easier to make it motivationally strong On the other hand 
if the situation is cognitively experimental it will be easier to set 
up with an adequate amount of control and precision The examples 
below illustrate the kinds of decisions which have been made on this 
question 


Lippm (23) m Ins experiment on the effects of the behavior 
tnpnf democratic leaders chose to make their expert 

mvamJ “""■'■'ely real for the subjects To do this he 

roms 111 “8*^ children into clubs which had their club 

as the sH , s laboratory The experimenter functioned 

Ulate the 1 r he svas able to manip 

coanitivelv reaT ''“'^hles Because of the desire to maintain a 
behavior were aU*' ’he possible variations in the leader s 

were produced w ^he differences between conditions that 

controfrnd urtr^' P°“‘hle that the lack of 

by havmg a'ccgniuv" l‘y" S XZ"' 

chose to hare f meniuvel”'’'?"''"' ’’^J'ction of deviates also 
the major measuref of re ^ s““ation for the subjects because 
responses to questions obtained from verbal 

would have more vahdire ,f f'b that these responses 

the part of the subiect rather in commitments to action on 

To obtain a cognitivelv answers to hypothetical questions 

students ^ situation he organized clubs of college 

ulation of variables the restriction on the manip 

consistent with the devised which were 

'^h high and low cohes?v^ness^^^ 

the degree of interest of th * investigator first ascertained 
and then manipulated th ^ ***^1^***^ ^n each of two kinds of clubs 
some groups of persons s.rt,^ ®*^ction to the group by composing 
and other groups of perso” highly interested in the activity 

type of manipulation of a"* ™ interested This 

satisfactory as other tf^r-hrT hy selection is probably not so 

techniques would be Because of the cognitnely 
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experiment which the investigator can communicate to the subjects 
and which they will accept. If this is not done, the subjects usually 
conjecture about it and make guesses as to the true purpose. If a 
plausible orientation is not given, this important aspect remains 
uncontrolled. 

The orientation which the experimenter gives the subject at 
the beginning should be plausible and should remain plausible as 
the experiment progresses. It is usually important that this plausible 
orientation not reveal to the subject the true focus of the experi- 
■ lent. The true purpose of the experiment and the true focus of the 
investigator’s interest can, and should, be revealed to the subjects 
at the conclusion of the experiment. 


TECHNIQUES FOR THE CONTROL AND 
MANIPULATION OF VARIABLES 

Since the basic purpose of a laboratory experiment is to achieve 
a simple situation in which certain variables can be vvell controlled 
while others can be varied at will, we shall attempt, in the present 
section, to be as detailed as possible. We shall illustrate not only 
the various techniques which have been developed or contro mg 
^ind manipulating variables but also the kinds of vana es w ic 
have been successfully controlled and manipulated in t le a oratory. 


Use of Pre-experime7ital Instructions 

The most obvious technique lor controlling or manipulating 
variables is the use o£ prc-cxperimenlal instructions to the subject. 
Such pre-cxperimcntal instructions vary greatly in their e ccti 
"CSS. It is probably safe to say that instructions to the subjects wil 
he successful in manipulating variables when these instructions are 
l-cpt simple, are given emphatically, and are plausible in the sense 
of being integrally related to the experimental activity in wliicli 
the subjects are to engage. The major dangers in the use o 
dons as a device for manipulating variables arc ( ) t le po 
maiicntion of the subjects when tl.c instructions arc given ami 
(2) the possible variability from subject to subject m ^ ' 

of the instructions. Because of these difTicultics, it is pro a i j 
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Perhaps the most frequenilj employed group activity is discus 
Sion Such an activity may be chosen uhen the purpose of the experi 
ment is either to study the involvement of people in an activity 
the amount of participation in an activity, or the communication 
or influence process that goes on in groups or to provide a relatively 
interesting activity vshich will involve the subjects in order for the 
experiment to accomplish some other purpose in the meantime 
Any topic which will be interesting to the subjects is suitable The 
Kcussion may concern differences in opinion, as in the experiment 
directed toward solving a problem, as in the 
enr«™^"^ J Ileutsch (9) or it may involve a sharing of expen 
as in the study of Festmger. Pepiione, and Newcomb (15) 
Quentlv hi ^ ^ subjects, a play activity may fre 

Ta al l (30). «hen he endeavored 

Ktoud nlav subgroups, had one sub 

subeVoup took"the'ror'ThT‘' 

actively emra^P,r helpers and servants to those who were 

autocratic and He " ^'^‘"8 1“" Lippitt (23), in his experiment on 
craft activities leader behavior, used various games and 

""" 'O »=hool age children 

group Kelley Mt'^har =' 'he activity tor the 

ducive to the estahb.l, ^ situation would be more con 

create a twX“r; 

subjects had to arrancr^ h which the 

tone (25), m an evn^r^ according to a certain pattern Pepi 

which was constructed'^"' w" productivity, used a work task 

relatively easy to obtain**^ ' treasures of production would be 

These are but a few of »h,. 

that can be prepared f many possible ex&mples of activities 

for the experimenters in'^ * There is almost limitless room 

best for his expenmental^^u"''^ ‘create a situation which will be 


ne Onentauon o, the Subjects 
Related to both the coim t 

activity in which the ero nature of the situation and the 

orientation to give the sub^ ** ^n&age is the problem of what 
sirable to have some Dlaus,M« * '"1 experiment It is highly de 
and understandable purpose for the 
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the personal goals that could be achieved through membership 
This was done hy informing the subjects that there was (or was not) 
a reward that would be given as a prize to each of the members in 

the best group , 

These instructions were probably moderate!) success u n 
one hand, they ^ve^e not integral to the experimental tas at is, 
the subjects could have done ever) thing the experimenter requne 
of them without these instructions ever having been given 1 e 
possibility of winning a reward or the likelihood t at mem ers 
would get along well with others in the group was, however, relevan 
to fairly important motives in the subjects They pro a y were 
concerned about whether or not they would like the ot er persons 
and be liked by them The possibility of a reward probably adde 
to the motivation to do well in the eyes of the experimenter 
results of the experiment show that a difference between ig an 
low attraction was created by means of these instructions 

In an experiment on the direction of communication i 
group, Festmger and Thibaut (12) wanted to 
ject s perception of the homogeneity or heterogeneity o t e^ p 
To create the perception of homogeneity, groups were to 
members had been carefully selected so that they were a 
same year in college and had equal interest in, and knowledge abou , 
the problem they were to discuss To create the , 

heterogeneous group, they were told that great i erences 
among them in their knowledge about and interest in, e p 
under discussion The manipulation of the variable by these instruc 
tions was only mildly successful Probably few of the subjects were 
much concerned with whether the group was homogeneous or heter 
ogeneous Although differences between these conditions wer 
tamed in the results, these differences were by no means stro g 
It might be expected that a more adequate manipulation of the e 
variables would have produced much larger differences between the 
conditions 

In an experiment by Festmger et a/ (H). ““™P‘ ''”^1 

*0 manipulate three ^arlabIes simultaneously, a tors 

instructions at the beginning of the experiment ^ * attraction 

'^ere interested in the interaction among the ° cxnerts in 

to the group, perception of \\hcthcr or not t icrc correct 

the group, and perception of s\hether or not t icre iv 
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sirable to manipulate more than one variable at a time through 
the use of pre experimental instructions Instructions which attempt 
to manipulate several variables simultaneously are likely to become 
so complex and so long that they render the manipulation inefFec 
tive We shall illustrate the problems involved in the use of mstruc 
tions y giving examples of successful and unsuccessful attempts at 
manipulating variables m this manner 

Deutsch (9) in his experiment on competitive and cooperative 
groups produced competitive or cooperative situations by differ 
groups In the competitive groups he told 
W "i? I be ranked accord.ng .o 

h .hr “ “'’""S “ the group and 

ankm^ Tf ftP^d m part upon these 

rltbuteH ™ •’■"h ‘"“t ‘he one tn that group tvho 

least would gefthe low®. contributed 

expenntenrml^Ae r,er .L "tb"' f 

compared witl, other grouns th!. ' m 

receive the same grade® and ®t°"P t™” 

by how well the ^ would be determined 

successful .rreatL tS ^hese tnstruc.tons were 

good examnle nf h ^ ^ required conditions and they provide a 

™..t ?h;"we“ret:rxT ^ 

tion of the situatmn k they provided essential explana 

subject and defined the' " '“hjccrs-they defined the goals for the 
reached tnanner in which these goals were to be 


cohesiveness comparing groups of high and low 

several kinds of motivti'"'''’' ’"‘“ction to the group by using 
in the members hiph *” some groups he wanted to create 

hking for the other s^traction on the basis of personal 

told the subjects that^°^^ ™embers To create high attraction he 
written down when th basts of the information they had 

group so that he was “"•^ered he had matched people in this 

one another To creat*^!***^ would be congenial and like 

of time scheduling diffi ^***^®^**®” be told subjects that because 
very well but that he unable to match them 

In other groups attra t *bink they would dislike each other 
c ion to the group was made dependent upon 
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was made to manipulate simultaneously three variables by verbal 
instructions to the subjects, a fourth variable was manipulated sue 
cessfully by means of false reporting to the subjects The subjects 
were to have a discussion among themselves concerning an issue 
about which each of them had already formed an opinion Before 
the discussion some subjects were given the impression that the 
group overwhelmingly agreed with their own opinion on the issue 
whereas other subjects were given the impression that the group 
overwhelmingly disagreed with them This was done in the follow 
mg manner Each subject wrote, on a slip of paper, his opinion on 
the issue which was to be discussed Subjects svere told that the 


experimenter would tabulate these and then give each person a 
tally which would show the opinion of each person in the grouji 
Thus, knowing everyone’s opinion, they would be able to procee 
sensibly with their discussion The tally which was hanoed to each 
of the subjects was entirely fictitious Each of the subjects in whom 
the perception of group agreement was to be created svas given a 
tally which showed all but one of the subjects agreeing very closely 
with him Each of those in whom the perception of disagreement 
with the group was to be created was handed a tally sheet which 
showed everyone in the group at least two opinion steps remove 
from his own opinion This false reporting proved successful in 
varying the degree of perceived agreement with the group 

We shall conclude the discussion of the technique of false 
reporting to subjects with an illustration of an unsuccessful attempt 
Festinger and Hymovitch (IS) attempted to create in subjects a 
feeling of rejection by the group Four subjects, strangers to one 
another, met in the laboratory and were told that they were to 
Work on a task which required cooperative effort although the 
various parts of the task would be divided among them They were 
first to have a brief discussion among themselves and get to know 
one another so that they could decide how they wanted to organize 
the task They were told that people who liked one another worked 
more productively together Consequently, if there was any one m 
the group that they disliked, it would be better to exclude that 
person from the group After the discussion, the subjects were given 
ballots on which each could indicate whether he wanted to work 
together with all the others or wanted to eliminate a member from 
the group If subjects chose the latter alternalive, the) wrote down 
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answer to the discussion problem This attempt to manipulate all 
three variables by pre experimental instruction was not very sue 
cessful The amount of instruction which had to be given to the 
subject and the complexity of the instructions rendered them rather 
ineffectne It probably would have been better to manipulate one 
of these \anables by instructions and to have devised techniques 
for manipulating the other two in other ways We shall discuss 
below such other techniques of manipulating and controlling 
variables 


Use of False Reporting 

False reporting to the subjects of the results of votes or of 
soaometne choices and the like is another technique for control 
an manipulation of variables Such false reporting must always be 
one in a manner which will make the report appear plausible 
trti^ ensure the acceptance of the report as 

\anables* ^ effective means of manipulating some kinds of 

Ca(hnir*"^^!i experiment on the voting behavior of 

niixed groups used the technique of false 
CToum ^ ^ subjects to keep the situation identical for all 
Se ^ members of the group voted for officers of the club in 
two manner There was first a nomination ballot to select 

received election The members of the group who 

tion Thi« ^ votes were to be the candidates m the final elec 
and sinr^ ballot was tabulated by the experimenter 

falsdv which ^ secret, u svas simple for him to report 

ner the evn*. members had won the nominations In this man 
the control which two persons were 

paid participrmwth"'*' a^pannient also employed 

"ere membe which will be elaborated below) who 

the results of^th By means of the false reporting of 

election m pv ^ ”°mmation ballot, the two candidates for each 
the two nnri ^ were two of the paid participants One of 
the other election identified herself as Jewish and 

thu election m 'each group 

^ standard situation 

" * e experiment by Festinger et al (14) in which an attempt 
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Pcpitone (24) reports an experiment in which he investigated 
the determinants of the perception of authority and approval in 
people. He was faced with the problem of how to provide a stan ar 
social situation for his subjects in which it would be meaningful 
to ask them for their perceptions of authority and approval. Using 
school-age children as his subjects, he let it be known in the school 
that, as part of a survey on interest in athletics, a three-man boar 
would arrive in a few days to interview many of the students. Those 
who successfully answered the questions asked by the three man 
board would win tickets to a college basketball game. The ^ 
man board which came to the school and interviewed students 
individually consisted actually of three paid participants ^ ^ 

been trained by the experimenter. Scripts for each of the three 
had been carefully written so that each boy who was interviewed 
was asked exactly the same questions. The responses to the oys 
answers were also standard for each of the conditions. In i eren 
conditions, however, the experimenter created authority differentials 
among the three board members and also differences among l em 
in the extent to which they openly voiced approval of the boy who 
was being interviewed. The boy’s perception of the relative author- 
ity and approval among the board members could be 
in an interview with each boy directly after his appearance befo 
the three-man board. Thus, the experimental situation was effec- 


tively standardized. , . . , 

Schachter (26), in his study of rejection of deviates, had three 
paid participants in each group. The topic for discussion was c ose 
so that all of the subjects would have opinions which very near y 
agreed with one another. Paid participants were used to crea e 
various conditions of deviation from this group 
participant voiced an extremely deviant opinion an le o i 
throughout the discussion. Another paid participant voiced a devi- 
ate opinion at the outset but allowed himself to e in »once 
>hat, in the end, he agreed with the other subjects. The th.rd pa d 
participant agreed at the beginning and continued to agree 'si 
modal opinion in the group. Thus, standard conditions ° ' 

lion from the group norm 'vere adiievcd and, by rotating . ^ 
Participanu among the various roles from group to ‘ 

“I'o possible to equate for personality factors. \\ e must c p . 
that these paid participanu had been very' carefully trainct 
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the name of the member they wanted to reject Each subject was 
then taken to a separate room and was told that the experimenter 
would tell him the results of the ballot as soon as possible Each 
subject was then privately told that the others had unanimously 
soted to reject him 

This false report to the subject was rarely successful The over 
w e ming majoritv of the subjects refused to accept it and imme 
diately suspected that the experimenter %vas not telling the truth 
ihe reason for the failure were probably twofold The experience 
t e ot ers in the preliminary discussion did not provide 

'he reported rejec 

lertc fT.H « I ^ ^ 1‘eport was unpleasant enough so that the sub 
renorf Many subjects refused to accept the 

or disbelief could not verbalize any reason for suspicion 

disbelief This technique had to be abandoned in this experiment 

Use of Paid Participants 

group and are Participants who are part of the experimental 

a relatueh p\nf>r, f'^’^tptilation of variables It is however 
pants When paid partici 

manipulate a vLiable K^'r'ound'"""^ participants to 

The same teclimoue L l “ “Penment by Sher.f (29) 

purpose (G) These exue others for the same specific 

laboratory so tJiat the l^ronght two persons together m the 

mnucnce the iudirmpnf!^If^ 'vlnch the judgments of one would 
subjects were asked to i * w might be investigated The 
of light This auiokineti” of movement of a point 

provided a rather imK. ^ ^ riot actually move) 

one ot the group m“ml ' experiments used as 

standard prearranged ser?,° ’r"'* Participant nho by making a 
a standard situation fo ii u ^““Sments iias able to produce 
tuecn Ills judgiuents nn,l H r** -Joels svith specified differences be 
gmeni, and the subjects .n.t.al judgments 
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belore it was delivered to another group member, so that the whole 
communication process could be reconstructed in the analysis. Aside 
from these measurement problems, there were other reasons for re- 
stricting the discussion to written notes. In an oral discussion, the 
person who is talking may be primarily addressing one or two others 
in the group, but, whether he likes it or not, what he is saying is si- 
multaneously heard by everyone. This introduces additional complex- 
ities. By limiting the communication process to written notes, with the 
further restriction that each note could be sent to only one person, 
the situation was kept simple and manageable. A further difficulty 
in using an oral discussion for the purposes of this experiment is 
the marked tendency for people to answer when remarks are ad- 
dressed to them. This is fully demonstrated by the usually high 
correlation obtained between the number of times a person com- 
municates to others and the number of times he is the recipient 
of communication (17). Since the experimenters were concerned 
primarily with other determinants of the direction of communica 
don, this would have been a complicating factor. The further restric- 
tion that the written notes could not be signed avoided this com- 
plication. The recipient of a note did not know from whom it came. 
The pads of paper on which the subjects wrote their notes were 
niarked so that later, in the analysis, the experimenter could tell 
'vho had Witten each note as well as to whom each note was 


addressed. , . . 

In his experiment on communication in a^ status lerarc y, 
Kelley (19) also restricted communication to written notes. Agai 
ihere were a number of functions screed by this restnct.on on e 
communication process. First, the experimenter .ntercepted all the 
notes Witten and thus had a detailed record of the commumrati 
process. Secondly, since all communication was by written notes, 
lire experimenter could easily manipulate the communication p 
«s. Actually, none of the notes which the subjects wrote to one 
another seas delivered. The notes which they received were fictit on 
one, designed to prodoce certain elfects. In this manner a standard 
pattern of receiving communications from othcre was 
for every group in all of the experimenter's conditions. 

Restrictions on the behavior of the group can also he produced 
fry an appropriate activity in whicl. tlie group '""“.'"PR'; 

aedviiy can be chosen to eliminate certain comphcaiions. 
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to behave in the group and in what kinds of things they could 
and Lould not say 

In the study by Festingcr (10) of the effect of knowledge of 
religious affiliation four paid participants were members of every 
group w ich met These paid participants were relied upon to 
variables and to create a standard situation In the 
middle of the experiment, when everyone was identified according 
er narne and religious affiliation two of these paid participants 
were h 'TV, Catholic and two announced that they 

ToZlrZ or Catholic were 

Dersomlir^*^^^^^ so that actual religious affiliation and 

ffiis man^e " -I’ ^e conditions In 

ences for n * powerful variables, which would affect prefer 
el R ous a^K^ -nd the effects of knowledge of 

The thr e r '""" 9-^^ ‘^’early 

in laboratorv examples of the use of paid participants 

sible range of uses demonstrate adequately the pos 

m planninff the hei, " \ experimenter and sufficient time 
training and rehearsinsThf^ end in adequately 

There is ample evident c™u Powerful effects can be produced 

non of variables with the^aidor'T ““‘P“'® 

la of paid participants 

Kertncn™ Beha.,cr Poss.b.lu.es 

ulate variables by crea^m^*^^ control over a situation and to manip 
of behavior ^ ^ s>tuation which restricts the possibilities 

Festinger and Thibaut n 9 \ 

terminants of direction f their experiment on the de 

the use of ivritten notes ° restricted the group to 

"■as made for a number^ t^frymg on their discussion This decision 
oral one, the direction discussion had been an 

I'ould have had to be r ° ^°”'*””**mation (who spoke to whom) 
the discussion was in observation of the group while 

groups IS difficult and observation in fairly large 

the use of written noiP. quite unreliable (see Chap 9) By 

able The exact timp record was immediately avail 

e note was written was recorded on if 
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exhaustively the various kinds of techniques for the control and the 
manipulation of variables. Those described are no more than a few 
examples of the wide variety of which an experimenter can avail 
himself. Many more possible techniques are likely to be developed 
in the -near future. It should again be stressed that when one em- 
ploys new techniques for manipulation of variables, or even some 
of those already developed, it is important to conduct preliminary 
experimentation to make sure that the manipulation is actually 
working. 


OPPORTUNITIES FOR MEASUREMENT IN 
LABORATORY EXPERIMENTATION 


Opportunities for collecting data in a laboratory experiment 
are present at all phases, from the recruiting of subjects until the 
find of the experimental sessions. There are, of course, some restric 
tions on what kinds of measurement can be employed at various 
phases in this process. These depend upon the design of the experi- 
n^ent and the way in which it is cognitively structured the 
subjects. ^Ye shall point out some of the measurement possibilities 
at each of the stages of a laboratory experiment. 

The first opportunity for measurement occurs before t le experi 
'genial session takes place. Such measurement may be made at the 
dme of recruiting subjects or when the subjects have assem c m 
rhe laboratory but before the experiment has begun. T le exact 
^inie at W’hich the measurement is done is immaterial an is gen 
firally selected for convenience. Such measurements, using ^ 
tionnaire or an interview, can have the following purposes: (1) to 
uhiain some measure which will be compared to a similar one taken 
during or after the experiment; and (2) to enable the experimenter 
control a variable by manipulating the composition of the 
S^uup according to these measures, 

In some experiments, it is essential for data to e co 
'fore ilie experiment began. Thibaut (30), in his expeninen 
colicsivcncss of privileged and underprivileged subgroups, c • 
^ uyed pre-cxpcrimcnial measurements to equate groups in 
'^M»crimcni and also to have a comparison between a prc-cxpcri. 
and a postexpcrimental measure. The subjects were members 



166 Research Settings 

the range of beha\ior, or produce certain reactions in the subject 
French (16) in his experiment on the effects of frustration and 
car on organized and unorganized groups, produced frustration in 
his groups by means of the activity in which they engaged The 
^ which was impossible to com 
p etc The frustration engendered m this manner was unmistakable 
nrifi rll experiment on the relationship between influence 
in esiveness, Back (3) wanted to produce a situation 

Z onl ^ together, had different interpre 

came 100 ^^ the same set of facts Before they 

and asked subject was given a set of three pictures 

"as actuallv I ^ subjects 

■nterpretatiLT''Th '‘fa’"" P'«“res, wh.ch would force different 
ever, were so slisht^h between the sets of pictures, hoiv 

seen d.fferem p.cturTL‘’th?s'' T"’’ 

ofaciivuy, Back^vas ^ manner, by appropriate choice 

be a difference of nn.r, f that, in every group, there svould 

of thetr discussion ^ ^objects at the beginning 

effectneness of different^^'"''”* '“''oagues (20) on the 

teehntquehas Lefr “mmunical.on in groups, a 

ample of renrict rn n " “ P”'’“P^ ‘he most extreme ex 

'sere concerned wuh ^‘^tiation In these studies the experimenters 
communication amon which of a number of patterns of 

effective problem soIvmirT"'**™^'’^ “ ^‘'’''P ‘‘““h 

munication, ihe exoer ® ° produce the different patterns of com 

rate to one another anT"?/’ “"owed some members to communi 
simple restriction on” prevented others from doing so By this 
'rere not available channels of communication were or 

h'hrff In these exoenmr,! .“’‘""'“"‘ration patterns were estab 
the artificiality of ihp purposes of the investigators and 

the subjects The restr*”f”'^^^*^'°" device were not hidden from 
that the subjects Ind situation, however, i\as such 

results of these exner. '’'**^*" as well as they could The 

ressfiil Such extreme ‘h"‘ ‘h= manipulation svas sue 

he appropriate, of cour of the situation svould 

problems a relatively selected range of 

faregoing discussion, wc have by no means covered 
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Questionnaires and intervietvs may also be used during the 
course of the experiment. These may take the artifiaa orm o 
questions interpolated into, and momentarily interrupting, t 
experiment or they may be disguised as election votes or expressions 
of opinion necessary to the conduct of the experiment. 

Schachter (26). in his experiment on rejection of devices, 
created a situation which was cognitively real to the su ject. e 
groups were clubs which the subject had joined an w ic 
subject expected would continue meeting perio ira y. t 
fitting, consequently, to ask the subjects to elect commitm^ to ca^ 
on various of the club functions and to vote on when an o^v o 
the club should meet. In this experiment, the data 
seen by the subjects not as such but rather as part o t eir unc 
ing as members of a club. _ , j 

In the Festinger (10) experiment on mixed Catholic an J 
groups, the major data were collected by holding e’ec"'”’* 
of a club. Here the situation was cognitively expenmen a 
subjects and the voting was undoubtedly seen as part o t ® 
mental procedure. The results indicate it to have been an a q 
method o£ data collection. 

One can also collect a wide variety ol data ’’5','!“ ,.«ion! 

interviews, or tests at the conclusion of the experime ' 

The techniques o£ such data collection are discussed in Chapters 

and 9. 


SUMMARY 

Laboratory experiments constitute a powerful 
investigating relationships among variables. le ess , j , 
«periLnu may be described as observing the eflect on n ^ 
triable ol the manipulation of an independent 
rontrolled conditions. Such experiments, if well d«ign ■ 
ducc clear and unambiguous results whicli may add to 

^Ucntaiion, as a technique for the dcs'clopmcnt o a p 
Ol knowledge, cannot List by i«lf. Expfr.nro"» - the lalmm.oo 
derive their direction from studies 
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of already existing clubs The investigator met the group at some 
designatedplace, usually their YMC A or their club He provided 
transportation for them to the experimental rooms Before setting 
out or the laboratory, he asked them to answer a questionnaire 
concerning who their friends were among the other boys He then 
brought them to the experimental rooms and was able to divide 
subgroups so that each person had about as many 
of his friends within his own subgroup as in the other subgroup 
“Paiment was concluded, the boys were again ashed 
tiaator ^ sQciometnc questions In this manner the inves 
friendshm w^il ^ equate his subgroups for amount of 

the elfecf of 'ih "" ' = basts for determining 

Mos of P^cedure on th.s vanable 

durinit the art ^1 P“^‘^dities for measurement occur, of course 
quently used P''°S^'^ss of the experiment One of the most fre 

discuss here some of the mh!‘ T'V" f 

possible during the Lperiment ' collection which are 

major sour«'*o7data' *" “ 

forms and mav hf* an take any of a variety of 

Kelley /191 m h ^ ''“‘'‘““a "'ays by the investigator 

hierarchy had hu u ®’‘P®'’'ment on commumcatton m a status 
the aoo/\„ acctVa„cr,“ T”"®' “»=■'" P^""" 

The actual product— th t^siruclions communicated to them 

the group fimsheH , exact pattern of bricks with which 

to obtain a measure f* ^^^tirded by the experimenter and was used 
In his exper?me„ production 

Deutsch (9) had the s^iSb ^®*^petitive and cooperative groups 
human relations probleiirH T""’ solutions to various 

of the group discussion”!^ tf analyzed these written products 
solution to the problem ° ^ ***” measures of the adequacy of the 
Closely related to s H 

subject makes in the nr”'” various records which the 

*n the Kelley n9\ doing the required activity Thus 

(12) experiment on direct”^*^ Festinger and Thibaut 

'vhich the subjects wrote ^h”l “"^nmnication, the actual notes 
main source of data ^ ^ on the discussion ivere the 
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and results must continually be checked by studies of real life sit 
uations The laboratory experiment is a technique for basic and 
theoretical research and is not the goal of an empirical science 
We have m this chapter, enumerated in some detail many 
designing laboratory experiments and for manipu 
a mg 1 erent kinds of variables in a variety of ways Many of these 
echniques for the manipulation of variables involve deception 
mveTrr" of subject, and the like As long as an 

emphaLe°\hrnecers.ty 'iTT 

to fhp eiiK j I y keeping m mind the responsibilities 
^ «penmenter must follow 

toIerZ hi -P“>nentat,ou ts fo continue and .s to b= 

form a service I ^ '1*'° experimenter per 

laboratorv pxne ° ™*>jccts m exchange for their help In all 
a full expUnatio^T!?, ^ f"*™ policy to give the subjects 

•.mes Sres sp"nrn; ^his sle 

tws with the group thannT discussing mat 

well the subjects kave fl i ‘e experiment If it is done 

and have not'^ wasted ih ' learned something 

been misdirecterand d “"If 

deceptions anruntrs^t 
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and results must continually be checked b) studies of real life sit. 
nations The laboratory experiment is a technique for basic ana 
theoretical research and is not the goal of an empirical science 
We have in this chapter enumerated in some detail many 
techniques for designing laboratory experiments and for mampu ^ 
lating different kinds of variables in a variety of ways Many of these ^ 
techniques for the manipulation of variables involve deception 
prevarication misdirection of subject and the like As long as an 
investigator works with human subjects it is impossible to oser 
emp asize t e necessity for keeping m mind the responsibilities 
e su ject and the ethics which the experimenter must follosv . 

experimentation is to continue and is to be 
fnrm ^ ^ people s\ho help in it that the experimenter per 

help In all 

a full ^’^penments it should be a firm policy to give the subjects 
t.mes expenment Th.s 

ters with the 'if® '”** “plaimng and discussing mat 

vdlT uhr^i ‘■'e experiment If tt ildonc 

nd h ve '•'« "'ey hate learned something 
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PART II 

Procedures fof 
Sampling 


Before we proceed from research settings to ‘h' 

data it is necessary to f”" j’ '"hrquestion Tf sam- 

now It is done and what its implications arc. 4 .ipridc 

pling may be simply stated: how is the ,,55 (or 

«hat persons or poups or organizations or commun. « o use fo 
•ho collection of his data? The way this decs.on ts m de wdl affect 
*lic conclusions which may be drawn and the pr 
conclusions. these 

Many investigators may J One docs research 

>!tcisions-that these decisions are made for the . .ol-i-rt, jn 

in industries into whicli one can get entree, one _ ^ 

laboratory experiments those ” “"be. do not obsiatc 

But such situations, frequent though they may oe. u 
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CHAPTER FIVE 


Selection of the Sample 


Leslie Kish 


There is no hard and fast rule for 
specialized work of “sampling” begins an “ convenient 

iised with vanous meanings in different con nhcprvations (or 

to exclude the processes and problems be emphasized 

measurements) from the area of this chapter u response 

that this exclusion is design {see Section 16) 

and nonresponse have bearing on the samp 8 decided on 

Assume, then, that a method of observation h"' „y ef 

whereby the value of some characteristic \vhich comprise the 

the N elements (members, individuals, case „rch is usually 

population (or universe) A target of information about 

tome numerical expression which summariz population- 

the characteristic from all the N elernen s f mch a parameter 

■ t . a parameter, a population value An example ot suen p 
-the mean . = i lx • Moreover, .he mean is an important and 
^ , Therefore, and for convenience 

eommonly known and used parameter , w, II center around it 

and brevity, much of the discussion to o oi pe mean will apply 
However, many of rhe general remarks concerning the mean 

also to other parameters observations have to be 

To obtain the exact value of a parame , 

*The number of elements in the populauon “ there are N separate 

* stands for the values of the variable charaetcruU ^ 

'^ues denoted by the general term x The symbol Sx stan 
''aiuei of X— that is, for Xi + xj xv 



the necessity for a consideration of the nature of the sample and its 
characteristics 

Sampling theory has made enormous strides in recent years, 
mainly m connection with the problems arising from large scale 
survey operations It is, consequently, easiest to talk about sampling 
m connection with the problems of surveys, and it is easiest to see 
the applications in that context But the applications exist else 
whe»-e, too, and they must be discovered and used As a single 
example, the reader may notice that the discussion of cluster sam 
pling in the following chapter is quite relevant to laboratory group 
experiments where perhaps twenty groups of six persons each are 
the subjects of data collection 
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employees’ W e obtain a (payroll) list numbered from 1 to 1 2, 
which each of the 12,000 employees appears once ^ ^ , 

random numbers, 400 different^ numbers are drawn (4 p 3 ) 
draw a five-digit number not greater than 12,000 is taken TheseJlO 
numbers will designate the numbers of 400 emp oyees e 
and other necessary identification are obtaine an 
viewers They are interviewed, and their answers are re 
numerical code ^ m 

Now we want to estimate the ° ^ 3 ,;.u,ate a 

the factory who hold a certain attitude, th , value For 

sample value to serve as an estimate of the popu a i simple 

nmple random sampling, the sample mean is 
and familiar proeedure of adding up all the values of the sample 
and dividing by their number it In symbols 

A proportion p is only a special kind of he value 

elements which possess a specified attribute are ' , j 

Of . = 1 and all other N(1 - .) ''--“idmg X-nit m the 
* = 0 The sample mean is obtained y ,r«hrr m the sample 

sample (r) possessing the attribute by t ® ^ interviews there 

= p' = r/a For example. ^ ^^'e^’^oportion/ = 80 - 

were 80 “yes” answers to a question T P 

WO = 20 IS our estimate of p. prop interviewed by the 

obtained had the entire population of 1 , a„Brceatc. of the Mai 

same procedures Moreover, our estimate o , ,, jhat question, 

lambir {Np) of employees who would have said y 
isNp' = 12,000 X 20 = 2400 n„iy one of 

The sample mean obtained from ® ® j j, ^ subject to 

“any possible values that could have c population value X To 
sampling variability We want to estimate me P ^ , ,o ^hich 

be able to do this, we must have a measure of the vanaoui y 

'By the word difrrmi we mean that it "j*' “““„ong the Hill unseleeted 

time Vic disregard It Thus, at each choice we are sel 6 

elements . , .u- samolc design was simple 

‘The subscript o is used here to ® designs wnll abo be 

'•hdom tamphng Similarly, eH.mate. detwed from 
^‘Snated by specific subsenpts 
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made on all the elements m the entire population However, seldom 
can information be obtained about all members of a population large 
enough to be interesting Usually practical considerations, particularly 
of cost, force us to be satisfied with making inferences about population 
values from data which are based on a sample only What are the tasks 
facing the researcher m planning a sample in order to make inferences 
about the population values^ He must (1) select a sample, (2) make 
observations, and (3) compute the s/afirfirr— that is, estimates based on 
the sample data 

These sample values are of little interest in themselves They are 
\sorth obtaining and are of interest to us only in so far as they yield 
information about the corresponding values m some population Thus, 
the selection of the sample and the process of estimation are two tasks 
which are incurred jointly because the population values are estimated 
from a sample Let us call the joint procedures of selection and estima- 
tion the sample design 

This chapter will concentrate on the less mathematical aspects of 
different procedures of selection The important but somewhat tech- 
nical problems of the use of different kinds of estimation will be largely 
neglected except for a brief discussion in Section 22 In the illustrative 
material the method of observation is assumed to be the interview 
Houever, the procedures have general applicability to a great variety 
of other procedures, situations, and problems 


FUNDAMENTALS OF SAMPLING 


1 A Simple Random Sample 

Bdorc proceeding further, let us look at an illustration of i speciBc 
mp c csign Suppose that it is desired to learn something about the 
attitudes of the 12,000 employees of a factory Suppose, also, that it 
method of sampling is to be “simple random 
samp ing and that the size of the sample will be 400 interviews ^ 

ou o \Nc select a “simple random sample” of 400 out of 12,000 


every “"cd sampling procedure whereby 

Sod mto the sample That duUoct 

lamohne’ the use of intervals of selection, wc call ‘ systematic 

sampling and discuss m Section 10 



Selection of the Sample 


179 


est se of = 983 X = 983 X 0200 = 0197 

The estimated standard error of the aggregate Ns can be given as 
est s e of {Nx') = N (est s e of X ) 

Hence, the standard error of the estimated 2400 employees who would 
have said “yes” to the question is 12,000 X 0197 — 236 emp oyees 
The simple random sample as described here is seldom use 
practice It occupies a central place m sampling theory ecause i 
serves as a standard of comparison and because it is the basis or t e 
various modifications of more complicated designs superimpose on 
In addition, its treatment here is justified by the fact that most o e 
formulas used in introductory texts, with which the reader is assume 
to be acquainted, refer to samples obtained by simple random 
The reader is warned that those formulas can not be use va i y i 
connection with samples obtained from other kinds of samp e esi 


2 The Sampling Distribution oj ike Estimate 

The information that the mean of a specific sample is 
intrinsic practical worth, Us value lies in that it te s us s 
about the population mean if Once we know that ® ^ .. 

20, just uhat can we say about je’ We know that the "vn . wiU 
differ from our known - 20, but xve do not know by how much 
The sample mean wdl depend on which sample of 400 ^ 

pened to be selected, since different samples of 400 persons would give 
different sample means The deviation from the 
any single sample mean is unknown, it may be arge ^ 

or minus The only fruitful way of looking at sampling van J 
terms of ^^hat size deviations are likely to occur in ^ ^ mean x' 

we ask gwen a sample design, what values of the sample mean 

*Thc discussion in this section concerns sample m general, 

design Here . without a«y-bscnpt denote -^Pj^ote any one of 
specifying the sample design The symbol obtained with the 

"" ™ 'h' distribution of all samplo moans which may ,5 „umalc 

•PWific dosisn (See, .on 16) The ‘ sample moan' hw' d'"o«^llm m 

be population mean It docs not necessarily re c _ -n for the means of 
“■"P'' m Sections 8 11, and 22 other “"Sphng doinbunon of 

other drs.gn. The discussion B relevant for the sampl. S 
listics other than the mean 
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i?/ IS subject (Sections 2 and 3) The standard error, the square root 
of the variance, of the sample mean is the usual measure of that 
variability For simple random sampling, the variance of the sample 
mean can be estimated from the sample as 

In 

est var of xj = (1 -/) — ^ where r* = r S(x - Xo'Y, 

n n — 1 

that IS, the variance of the sample cases, with (n — 1) used as divisor 
(The statistic j* is a sample estimate of the variance of the elements in 
the population ) 

This formula holds also, of course, in the event that our mean is 
a proportion However, m that event there is a form which sa\es the 
labor of squaring 

est var of^' = (1 — f) ^ ~ P') 

n — 1 


It may be noted that here /.'(I _ p’) takes the place of its 

equivalent r* 

These formulas will appear familiar to the reader with the possible 
factor (1 — /) which for a simple random sample is 

/) “ (1 “ n/iV) This ‘ correction for a finite population’* arises 

w en sampling from a finite population “without replacement “ This 
ast p rase refers to the procedure which prevents any element from 
oeing selected into the sample more than once (We did this by 
se ccting 0 diferent random numbers, not allowing the same number 

0 appear twice ) In most practical cases the sampling fraction (n/N) 
so sma t at this correction is of no importance In reading the 

o/the should slip past it and on to the important parts 

1 /in* T?' *<= sampling fraction is ti/iV = 400/12,000 - 

1/30 Then (1 -/) = 1 - 1/30 = 29/30 = 967 On the standard 

eni?Jl bemg equal to Vl - 1/30 which is closely 

tn iimu, ^ 1 ~ nases this factor is so close 

illnstrat ^ I'P^Vmg by it has no appreciable effect In our 
illustration we have for p' = 20 


est var of 5/ *= 967 ( 80) 

399 

Its square root IS 0197 Or, again 


967 X 000401 = 000388 
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The deviation of each possible sample is taken f™” *e 'ne^' 
the sampling distribution It’ These deviations may “ 

(*.' - n Then the variance of it’ around is calculated by taking 
the sum of the squared deviations each multiplied by its probabih y 
Its “relative frequency”— of occurrence 

var (X') = 2 P. (*.' - X'Y 

(The summation is for all possible samples ) 

The standard error of the sai^le ^ d“tnbutm rV'- 

seats the random fluctuation, the variability , measured m 

to a specified sample design The amount o 

terms of the standard deviation of the samp i g sample mean a', 

tmportam quantity is called the standar „ ,5 always 

and we shall abbreviate it as se of . ,nbution of all sample 

defined as the standard deviation of the 

means under a specified sample design nractical work to 

Now, as we have said above, it is not ,he dis- 

obtain the standard error direcUy from the actual tab^^^^^^^ 

tribution However, through the use developed specific useful 

distnbution, mathematical statistician sample means for 

formulas for the variances and stan ar developed prac- 

many practical sample designs Moreover, ^^^d standard 

tical formulas for obtaining esttmales o calculated from the 

errors of sample means— estimates w ic eiven for a simple 

data of a singie sample One example is the formula given 
random sample 

est se (^/) = ~ ^ ^ n 

The baste dejimiion of the standard ^ for that specified 

sample design i( is the standard error of the samp m sample designs 

^^fnple design But it must be emphasize t a error, some of 

fiacre are different formulas of the estimate different sample 

*esc will be given later m the section rorm of confidence intervals 

designs They serve as powerful tools m the form of conn 
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are possible and what is the probability of occurrence of each of those 
values'^ The array of possible values of JE', each with its probability of 
occurrence, is a distribution, it is the sampling distribution of the 
possible sample means x' + 

In the course of empirical research, when we calculate the mean 
from the single available sample, we cannot obtain the actual values 
for all the sample means possible under the design Nevertheless, the 
distribution of all possible sample means is an important theoretical 
concept It is what we should have in mind when we think about 
sampling fluctuations, sampling errors, standard errors, and such 
Hence suppose that there has been specified a sampling design to be 
applied to a given population — including the size of the sample, the 
method of selection, and the method of estimation Now imagine that 
all the sample means (5,') possible under the design arc calculated, 
and so is the probability (P,) of occurrence of each of those sample 
means ^ This probability is analogous to the “relatne frequency” m 
the calculations of the common formulas for variance Now we have 
the sampling distribution of the different values that the sample mean 
A', may take The mean of that distribution— i e , the mean of the pos- 
sible sample means-is denoted by x% and in well designed samples 
X is either equal to the parameter X or is close to it The serious 
problems of the differences between x' and X which arise in practice 
are discussed briefly m Section 16 Until then, we shall ignore the 
differences between the parameter a: and the mean of the sampling 
distribution X " * 


tSec (18 p 2 4 4 p yj 43 and 16 p 95 103) 
drawn was speafied sample after sample was 

After many « sample was calculated and these values were tabulated. 

beSme ZreSu T'l distribution would gradually 

distribution woi M * « number of samples drawn increased, the shape of the 
tion that for mo« sampUng distribution In Section 3 we men- 

IThese diff *®niples that distnbubon is close to a normal distribution, 

owm^ to n™"" r*. great practical importance. 

Hence m this champ'' th "on»a«npling errors of response and nonresponse 
pStmeirs bur ' staten«nt, of statut.eal inference are made not to the 

have been obtained if th° values ’ These arc the values that would 

rather thaa oaly a r™p”(fcn™ In* The'"'’ ’’"h ‘'“'8"““'“ "teervaaon 
errors of re.nonL ^ \i»ec«on 16) They arc subject to the same sources ol 
They mavTZaJh “ ”>“<* the saarple ernmate, I.' tar .abject 

SrpirSimate. “ <« the mean s ' of the dt.tr, button of the 
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ic) In the illustration in Section 1 we calculated the 
as .20 and its estimated standard error as .02. Now we make he 
statement that the population value lies in the mterval between the 
value of .20 - 2 (.02) and the value of .20 + 2 (.02), that 
16 percent and 24 percent. This statement ts the result “f *^p 
we happened to draw; on the basis of another 
said between 19 and 25 percent, or between 14 and p , 

The probability that statements of this kmd are eorrect is 0_ 95_ ■^at ^s 
in the long run, about 19 out of 20 statements of this bmd 'v >1 
correct. tLs, there are 5 chanees in 100 that the 
turn out to be incorrect; that, if we obtained the population value, 
would find that it lies outside the limits we set { , PP' _ 

In general, we make the statement t at the P°P— 

somewhere between the value of a: ^ ^ awertion 

meat will have a 95 percent probability of being 

IS correct. Here t is the normal devi , 
making correct statements, are: 

3.00 4.00 


.67 

.50 


1.00 

.68 


1.96 

.95 


2.58 

.99 


.997 


.99994 


1 t.>n the truth increases rapidly with an 

The probability that ,he length of the 

incre^e m the value of 1, that .s ,.onfidence interval, the 

confidence interval. ® ,he level of probability at 

less useful it is. It is general P”^ „ t ,he length of the 

some point and *en use ® ,„el of probability 

interval. In social science ^ ^ chapter, the value 

IS used, corresponding to a value of 

of 2 b used as an ihc confidence interval without 

Our aim IS to reduce me lenh* Thr reduc- 

decreasing the probability of goal of sample design, 

lion of the standard error of sample resu B 

discussed in Sections 7 and 14. 
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3 Confidence Intervals 

With regard to the use of confidence intervals for sample estimates, 
ome understanding of several important points is necessary 

(fl) From the result® of \\ell»planned probability samples, it is 
possible to compute values for the estimated standard errors of sample 
means Mathematical statisticians have derived formulas from which 
we can compute those values or some useful practical approximations 
The standard error is defined as the standard deviation of the distribu- 
tion of the sample means, and the distnbution depends on the specific 
sample design used Therefore, the formula for the estimated 
standard error will depend on the sample design used In the case of 
si mple random sampling, we have the familiar forms of s/ \/n or 
y/p — p) i {n ~ \) But these formulas will not hold for other 
sample designs The difference may be either in the selection or in the 
estimation procedures 

(i) Many of the estimates used in practice have sampling distribu 
tions w ich are approximately normally distributed Just how good 
t at approximation is depends on the underlying distribution of the 
characteristic in the population and on the size and design of the 
samp e or any variable encountered in practice, the approximation 
improves with the size of the sample 

h- i?” ^ ^rm\[ sample, or some other reason for doubts, 

s ou ascertain whether he may proceed with the assumption of 
norma ly If the sampling distribution departs senously from the 
c j kinds of alternatives are open for the construe 

tion ml? ^ search may be made for some distnbu 
n other than the normal, to serve as a useful approximation Or he 

° ® “distribution free” statistic (see Chap 12) 

work samp e results encountered in practical social research 

comoareH normality will lead to errors which are small 

assumntion ” f” of inaccuracy which are tolerated The 

rrorhTh "T f statements of probability 

ihn™' , H "1-““ - Wo-- 

Iherebv a-jen ?[. ^ ™akmg those probability statements, tsc 

to the Lmplmg disTribmio™‘‘‘ ^ approximation 

non oof ; oistribution of our estimates In so far as that assump 

wLodiff ■''' r ’ wiU have a probability of being 

wrong different from that which we intended and stated 
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congruent with the probab.hty model that underlies 
theo^ Terms such as “tossing perfect coins” or 

WeLtLTp“at:^^^^^^ 

to tossing, mixing, and drawing t problem is 

want to deal-the elements themselves might “Hect i P 
met by listing and numbering the elements, and *en mmng and 
drawing uniform objects bearing their „ 

what IS supposed to go on in a bingo ‘=7’,°baih or perfect urns, or to 
difficult to construct perfect corns ^urvalent of that process has been 
bring about complete mixing The q 

performed by careful experts 7“ complete mixing of 

random numbers -our “7"'7e mechanical process of selection, 
perfect balls Thus, we see that he mechani k 

which IS indispensable P''°7'*'*’^elect^ properly from a table of 
following Cham ^ J„,b«s on a lisung of individual 

random numbers units the identification is made to a 
sampling units, from these seiectea 

set of physical units which ™‘‘ 'Xroblem^nvolved in the identifica- 
Sometimes there are P«<=“ ^ ^ ^ numbers on the 

tion of the -dividuals associated wit^hej^ 

listing These arc problems of P j merely assumed 

«th clear, din, dying employees from a payroll list. 

Sometimes, asm the "f *= 7 other times, as m identifying 

the task may be simple and easy orocess may call for skill, 

dwelling units from a block listing sheet the process may 

and mistakes may occur (see S"=“on 1 

Furthermore, there are some ^ 

the making of a ^uftfaetory I7 P 

may be incomplete, for u'umple, employees hired since some 

may exclude white-collar em^, ^P^' explicitly from the 

recent date It may be dccid^ ^.^Wish a separate stratum for them 

population Alternatel>,onemay bl h^^^^P through 

80 that they will obtain the proper p Y 

separate sampling procedures 
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4. Measurability The Need for Probability Samples 

The use of confidence intervals is based on statistical theory. The 
application of this theory can be developed only for those samples in 
which the probability of selection of every element of the population is 
known There is a gulf between the known sample result and the 
unknown population value, the confidence interval is the only objective 
statistical bridge across that gulf Confidence intervals are based on 
the proper estimates of the standard error. But standard errors may be 
calculated validly only for probability samples— .that is, for samples for 
which the probability of selection of every element in the population 
IS known * Therefore, if we wish to make use of the theory of statistical 
inference, we must use a probability sample. 

The property of a sample which enables the researcher to make 
estimates based on sample data of population values and then to 
calculate confidence intervals for those estimates has been called 
measurability It is desirable to consult a sampling statistician with the 
p ans eJoTt the survey to sec whether the design will allow the valid 
calculation of the precision of the sample. There may be other ways of 
judging the adequacy of samples, but they depend on personal judg- 
tnent a nonprobability sample is taken, such as a “quota sample” or a 
typica city, the results may be good or they may be poor. But 
^atistical theory is lacking for determining the accuracy of the results. 

* occasions when for a small informal sample one can 
th ‘Spensc both with precision and with its measure. However, 

IS c apter wi I be confined to the problems of probability sampling 


5 Mechanical Selection and the Use oj Listings 

select a probability sample? Whenever a unit is 
be maHe h ° ^ from among other units, the selection must 

orobabilitie<!^°^*^ ^cchanical procedure which guarantees the desired 

■nvolved Wc ne=d some 
physical operation, some practical procedure, which will be reasonably 

onymoualy with jwiaiiS authors syn- 

15 used, we shall mean ^ n ^ ^ Chapter, when the phrase random chout 

dtfioed group of sampUng Smo'” "■* equal probabdity among tht 
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m the various phases of area sampling, presented in Sections 19, 20, 
and 21 Of particular influence is the size of the sampling unit used in 
the listing which results in clustering, subsamphng, and multistage 
sampling (3, pp 76-87, 22, pp 60-80) 

Area sampling is an important kind of listing procedure because 
It IS used widely in social studies It is also used m other types of surveys; 
for example, crops or other flora, as well as grocery stores, have been 
sampled with the use of area segments Its widespread use in social 
surveys is due chiefly to the relative ease of identifying each member 
of a human population with one, and only one, dwelling unit In turn, 
these dwelling units are identified with area segments, also uniquely 
Thus, a selection of the area segments yields a sample of dwellings, and 
these in turn a sample of people It may be expected that the unique 
identifications of people with dwellings, and of these with segments, is 
troublesome and imperfect This becomes a practical matter of doing a 
good job within available resources In this connection one may men- 
tion the necessity for boundaries which are clear, unambiguous, and 
easily identified in the field 

7 Precision, Variations in Sample Design 

It IS our general goal to obtain as small a confidence interval as 
we can for some fixed level of probability of making correct statements 
The smaller its confidence interval, the more useful is the sample 
estimate For a fixed probability !c\el, the length of the confidence 
interval depends on the standard error For this reason the \Nord 
brectsian is often used to denote the inverse of the standard error (or 
sometimes of its square, the variance) of the sample estimate 

The standard error of a simple random sample vs sj -s/n Within 
the limits of that design, the way to increase the precision (to reduce the 
standard error) is to increase n, the number of elements in the sample 
The standard errors of other sample designs arc different, but thc> all 
have the common property that to increase the precision we must take 
more of something (persons, dwellings, blocks, counties, etc ) But to 
take more of anything costs monc>— and generally there is a limit to 
what the sample ma> cost Tlic question ma> be asked, then For a 
guen expenditure, how can wc get the greatest precision’ (Sec also 
Section 14 ) 
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Secondly, some elements may appear more than once on the list 
Perhaps employees who work m more than one department will have 
more than one payroll card If these individuals are to have the same 
probability of selection as the others then one should eliminate the 
duplication* through the entire list If this would be too difficult, some 
other adjustment has to be made m our procedures 

Thirdly, a list may contain other items m addition to members of 
the population Some lines may be simply blank, belonging to no 
units, and others may belong to units of some other and excluded 
population Let us say that our list identifies the 12,000 employees of the 
factory by their actual payroll numbers These are not consecutive, 
but run irregularly up to 99,999 Thus, among the five-digit numbers, 
there are 88,000 which do not belong to any of the 12,000 members of 
our population Some may be blank and some may belong to office 
employees, whom we want to exclude from the sample We simply 
raw ve digit numbers from a table of random numbers We inspect 
t c ist or each drawn number if it belongs to a member of our 
population, we have a seleclion, otherwise, we have not We continue 
this process until the designated size of the sample (n) is reached This 
gnes us a simple random sample of n elements out of the designated 
popu ation n general, the list should be inspected carefully before 
rawing to take whatever measures are necessary for the insurance of 
the proper probabilittcs of selection 


6 Lisltngs and Ana Sampling 

available for the selection process is an 
do ■" ‘*"■8" of F'^otors which are 

rov er?!, ‘n of '''‘od sampling units, the extent of 

auxiln?’ r' “““ratty and completeness of the list, and the amount of 
sec forstrl'f ***• This last factor is useful, as we shall 

Those **^**°"’ "Measures of size, and in the estimation process 

of the den T f determine the nature of the sampling design and 

of the details of the practical selection procedures 

rnmnleie 1.®^^ ®^*”phng Situations there does not exist a simple and 

Section ^ '"^"'‘duals, such as the factory payroll discussed 

imnomnt n " °^“'er, it may be too costly to construct one The 
important practical complications which a list may possess are present 
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Attached to many formulas in textbooks is an important but often 
overlooked phrase which reads something like this “Given n inde- 
pendent selections made with equal probability ” Actually such a 
selection is seldom given to the researcher Furthermore, in practical 
social research such a selection is seldom taken by the researcher 
Therefore, the automatic use of those formulas which assume simple 
random sampling is not justified Sometimes their unjustified use leads 
to the construction of confidence intervals which are much too narrow 
The result in those cases is that the estimate will lead to incorrect 
statements considerably more often than the researcher would wish 
(Section 13) Serious mistakes of this kind are frequently made, owing 
to the disregard of the clustering present in the selection process 

The proper calculation of the variance of a sample estimate must 
be done in accord with the procedures used in the sample design In 
order to call attention to that need, a specific subscript was used to 
denote each sample design Thus if/ denotes a sample mean obtained 
from a simple random sample (Section 1), if'prop denotes the mean of a 
proportionate stratified sample (Section 9), and so on 

One word more of caution may be appropriate here What may 
appear to be a slight change in the sample design may affect senousiy 
the variance of the sample estimate Furthermore, some change in 
procedure may introduce a serious bias into the design 


STRATIFICATION TECHNIQUES 


S Stratification 

Stratified sampling is the procedure of dividing the population 
into subpopulations, called strata^ and then selecting a sample within 
each Every sampling unit m the population is placed in one (and only 
one) of the strata prior to the selection of the sample so that the sum 
of the strata is identical with the population Wilhtn each stratum, a 
sample is selected from the units m that stratum, and from each of those 
samples the estimate is calculated for its stratum Finally, the separate 
estimates for each stratum are combined to form an estimate of the 
total population value (However, the simpler calculation is available 
for the means of proportionate stratified samples, given in Section 9) 
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Now we calculate 

= 6 X 20 -}- 3 X 35 + 1 X 45 - 27 

That IS, 27 percent is the combined sample estimate for the proportion 
who would have answered “yes” to that question if all the employees 
of the company had been questioned 

The estimated variance of that sample result is 

est var = ( 6)* ( 020)* + ( 3)* ( 020)® + (1)*C040)*= 000196 

That IS, the estimated standard error is 

est se (sj) = V 000196 = 014 

Therefore, with the use of two standard errors, the statement is made 
that the percentage of the entire 20,000 employees of the company 
who would have said “yes” to that question is between the limits 27 ± 
2 ( 014), that IS, between 24 2 percent and 29 8 percent This statement 
has a probability of about 0 95 of being correct 

Note the important implications of the above From each of three 
subpopulations a sample was chosen separately The designs, the pro* 
cedures, and the sizes of the samples in each of the strata are unknown 
and irreveiant We have the sample means and their standard errors 
as they were calculated separately from the three samples Now it is 
desired to make an estimate for the combined population represented 
by the sum of the three subpopulations (strata) The relative number of 
elements m each of the three strata is the weight of that stratum, with 
the use of these weights, the separate stratum estimates are readily 
combined to obtain estimates for the entire population 

There are three kinds of reasons for using a stratified sample 
(o) Stratification may be aimed at the reduction of the variances of 
the sample results for the entire population Thereby greater precision 
IS obtained for the sample estimates, a constant goal of sample design 
Section 9 discusses one method of aiming at that goal. Section 11 
discusses contrasting methods (1, pp 65-110) 

(6) It may be thought convenient or necessary to use different 
sampling methods or procedures in different strata of the population For 
example, m one of Uic factoncs of the illustration, a sample of indi- 
viduals may be selected, m another, the sample mi> be selected in 
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The process of selection in each stratum is earned out separately 
and independently In each of the strata one may use a different 
sampling fraction, and even different methods and procedures Regard- 
less of the procedures used within the different strata, the strata means 
may be combined to form an estimate for the population thus 


a 

Xj = JiWh Xk' 


That IS the sample mean Xk obtained separately for each stratum 
IS multiplied by the weight of that stratum, then these products arc 
summed over the R strata to obtain the combined weighted estimate 
Xu. or the entire population The weight of each stratum is the 
proportion o the total population contained in the stratum The sum 
of these weights is 1 that is, 2uf* = 1 

The variance of the combined weighted mean will be the sura of 
e variances o t e individual strata means, each multiplied by the 
square of the stratum weight 

est var (V) = L** (est var (j/)] 

IS onW^onp nf Imagine that the factory mentioned in Section 2 

of attitiiH^-s j ^ company and that a separate survey 

ICthCr ■" O"' questions appeared in 

of all emnlnv 't is now desired to estimate what percent 

employees of the company would have said "yes” to this question 


Stratum (factory) number (A) 

1 

2 

3 Company total 

Number of employees m stratum 

12 000 

6 000 

2 000 20 000 

Relative proprortion of em 
p!o>ees (ws = 

06 

03 

0 1 0 

Sample mean (the proportion 
of yes ans\vcrs) for stratum 
(^a') 

20 

35 

43 

Estimated standard error of 
sample mean of stratum 
[est sc (JaOI 

020 

020 

040 
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However, the information used for the sorting of units into strata need 
be neither objective, accurate, nor complete, on the contrary, personal 
judgment can often be used profitably for this purpose 

The role of stratification in sample design is often misunderstood 
and exaggerated Sometimes it is implied loosely that if stratification 
IS used we may shut our eyes to other aspects of the selection procedure 
This implication is used to justify the use of nonprobability methods of 
selection, such as “quota” samples This attempt to establish stratifica- 
tion as a sufficient condition for an adequate sample design has no 
basis in statistical theory Each stratum is a subpopulation, and the 
pnnciples of probability sampling must be applied to the selection 
procedures used within the strata 

Sometimes it is implied that stratification is a necessary condition, 
a “must,” for an adequate sample design It is far from that Actually 
the sampler usually has to be satisfied with relatively dull vanables 
(such as age, sex, etc ) because the more interesting vanables (such as 
basic psychological make up or social history of the individual) arc 
not available In many practical cases the gain from stratification is 
little, that IS, the same precision may be achieved with but a little more 
expense without the use of stratification Nevertheless, stratification is 
used in most sampling undertakings because it is generally beneficial 
and because ic is easy to apply 

9 A Proportionate Sample of Elements 

Proportionate stratified sampling of elements is often in the back 
of people’s minds when they talk of “representative sampling,” uhen 
they insist that the “different parts of the population must be properly 
represented ” Let us describe a propoTtionate stratified random selection of 
elements by means of an example Suppose that we want a sample of 
n = 400 employees out of theiV = 12,000 employed in a factory We 
suspect that there may be important differences in the attitudes to be 
measured among the employees in the different departments There- 
fore, the employees are listed separately for each of the four strata 
(departments) into which the company was divided 

In order to have a stratified sample, the selcciicn must be earned 
out 5cparatel> m each stratum It is a selection of elements because the 
elements (employees) arc selected indi\'idua{I>, separately In order to 
make it a pToportionate sample, the number of elements from each 
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clusters of work sections These differences in procedures may be in 
accord with differences in the physical distribution of the population 
elements or with differences in the manner in which they are listed, 
or they may be due to differences m total survey objectives within the 
different strata 


(c) Sometimes strata are established because the subpopulations 
arc also designated '^domains" of study — that is, the survey is designed to 
provide sample results of some desired precision about the several 
subpopulations separately, as well as about the combined whole In 
our illustration, the sample results (jf*') were desired for each of the 
three factories, the domains of that study, and the desired precision 
(s c of Xk') was used to determine the design and the size of the sample 
to be taken m each 


What characteristics of the sampling units of a population should 
be used for stratifying the units m order to reduce the variances of the 
sample estimates for that populauon> 

stratifying charactcnstics should be related to the variables 
0 bt Btimated from the study The sorting of the sampling units on 
Will ° ^ ® Stratifying characteristics should establish strata which 
tn K sorted with regard to the variables 

reduction of the variances of the sample results is 
variation (of the characteristics studied) 
® the strata is less than their vanation 

maVe. ^ population Hencc, m stratifying, one attempts to 

possible ^ within the stratum as homogeneous as 

lust wWh wasteful to spend much time worrying about 

usuaMv h/"" “ ^ Experience shows that 

if both ar Uluch difference in the precision of two procedures 

oerson arnM«. ou some reasonably good stratifying variables A 
reasonable rh^ ^ T**** sul^cct matter will usually hit readily on a 

brine comm " elaborate and expensive further search may not 

Dring commensurate gams 

( ) Each sampling unit must be assigned to one of the strata 1 


Uncouj stratum is davutd Lmat.TO “d u' °' '■ ’*“'’'‘”5 ™‘“- 

formauou cheaply m Utc first ptSlTfor ““P^g ' u used « “htam m- 

secoud phase, a smaller sample u^^SsdieS fcTh* “ “"F u d ‘?1 ^ 

268, 22, p 38 , 16 , p 153 ) ^ for the mam part of the study (1, P 
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Assembly 

Foundry 

and 

machines 

Office 

Other 

Entire 

factory 

Stratum number h 

I 

2 

3 

4 

Total 

Stratum weight Wh 

333 

250 

250 

167 

1 00 

Number of employees selected 
from stratum «a 

133 

100 

100 

67 

400 

Number of “yes” answers in 
stratum Xf 

12 

11 

36 

21 

80 

Proportion of “yes” answers 







09 

11 

36 

31 



tth 


Because the sample is self-wcighting, the mean for the sample may 
be taken simply as (he total for the sample divided bv the number of 
cases in the sample 1 

^pri>p “ 

n 

In our example, we have simply 80/400 ** 20 The subscript “prop” 
of the sample mean identifies the sample design with which it was 
obtained The term self-weighting denotes that, in calculating the sample 
mean, the sample cases are simply added without any special weighting 
procedure 

Vi-ai "h ^aTf»'p\e 'wViyc.Vi TiWi ■pitypw'nOTiait, 

sample would not be self-weighting (Sections 11 and 22) In that case 
the mean would have to be obtained by a weighting procedure 
Xf = ZwhXf = 

333 X 09 + 250 X 11 + 250 X 36 + 167 X 31 = 0 20. 

« 1 "* 
•/\s gum in Section 8, «» ZukXk The mean for the stralumu Xi' *“ — 2 x 1 

n* 

* ni 1 1 

In a proportionate sample, ui «■ ij»/b Therefore, = 2— — 2xi ■» — 22x» 

n fli It 

But the double summation means simply that after the sum of all values in each 
stratum u obtained, these partial sums must be added for all H strata to obtain 
the sample total This summation can be done all in one step and ^vc ma> express 
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Foundry 



Symbol 

and 

Assembly machine 

Office 

Others 

Entire 

factory 

Stratum number 

h 

1 

2 

3 

4 

Total 

Number of employees 

Nk 

4,000 

3,000 

3,000 

2,000 

12,000 

Relative weight (Nk/N) 

v>h 

333 

250 

250 

.167 

1 000 

Number of employees to 
be selected (jiua) 


133 

100 

100 

67 

400 


stratum in the sample must be made proportionate to the number of 
e emcnts from each stratum m the population The numbers of elements 
m each stratum relative to the population total (N) is denoted by the 
stratum weight w\ ~ Nh/N Now if we multiply the total desired size 
o e s^p e (n) by the weight of the stratum, we obtain the number of 
strata n\ = nw/, (as shown on 
♦ . hus the sample will be proportionate because the 
that °u stratum in the sample is equal to the ratio of 

4000/12^'^ 33?°’’“"'“" Forexample. 133/400 = 

samr^mu^f regarding a proportionate sample is that the 

the nonitl stratum is equal to the sampling fraction for 

* /30 Thai is, the sampling fraction 

asmau Jiff r ^ “ 1“^'''''’® = ’“/SOOO = «/2000 (There is 

usuallv in r 'r'tprecise fraction in this case, as there is 

samn ml r a => '""al matter) That is. the 

“ V30 IS obtained, then this 
There 7 “f dements (IV,) in each of the strata 

sample desien "'ord to be explained in our definition of the 

the n, elements °ourof tU77 

separate ranHorra u * each stratum is to be made by a 

ments lusi as d fi equal probabilities among all the ele- 

*7ent Idom 077 7“°" ’ '■ ^r example, 133 

of random numbers "tS ^ 

random sample t ' stratum we select a simple 


lon. Thu u morc*^ua°rn*pracLr^ ^ described in the next 
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formula belongs to simple random sampling (Section 1), a sampling 
method different from the one wWch was actually used. In the present 
case the difference is not great* but in Section 1 3 some great differences 
due to clustering will be noted. 

The ratio .000354/.000388 = 91 percent is a measure of the 
9-percent gain in precision over simple random sampling which is due 
to the stratification in this case. That is, it would take about 400/.91 = 
440 interviews by simple random sampling to get the same variance as 
the 400 interviews in the proportionate stratified random sample of 
elements. Several things may be said about proportionate sampling 
of elements: 

(a) The gains in precision arise because each stratum is propor- 
tionately represented in the sample- This helps by eliminating from 
the variance of the mean of the combined sample that component of 
the variation which is due to differences among the means of the several 
strata. Another way of looking at this is to note that in this type of 
sampling the variability arises only from sampling within the strata. 
In so far as the strata are homogcncous—that is, in so far as the varia- 
bility within the strata is less than in the population at large-just so 
far will the stratification be useful. 

(b) The gains from the use of proportionate sampling of elements 
arc usually not spectacular. In the example the gain was 9 percent, 
which may be looked on as a 9>pcrccnt reduction in variance for a 
given number of interviews (about 4j/£-percent reduction in the 
standard error); or as a 9-pcrccnt saving in the number of interviews 
needed for a fixed variance (sec Section 14). Actually, this saving is 
greater than one usually finds in practice and it is due to the remark- 
able differences of the proportions (p/,) in the four strata. Let us say, 
for example, that the proportions holding for another attitude on the 
same survey are 40 percent, 50 percent, 50 percent, and 60 percent in 
the four strata; the gain from stratification for that characteristic is 
only 2 percent, the equivalent of 8 interviews in 400. The gains from 
stratification may be shown by simple formulas (3, p. 227 ; 8, p. 1 1 ) . This 
modest gain is in sharp contrast with rather widespread, vague, notions 
of stratification. Perhaps some understanding may be gained from the 
consideration that without any stratification a simple random sample 
would obtain in most cases nearljr the correct proportions from each 
stratum. In our e.xamplc, a proportionate sample allocates 133 out of 
the 400 interviews to the first stratum. Without any stratification a 
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simple random sample would obtain some number between 124 and 
142 two thirds of the time, and between 114 and 152 m 19 samples 
out of 20 

(c) We shall see in Section 11 that in some cases bigger gams may 
be obtained from allocations of the sample which are not proportionate 
\nd in Section 22 an example is given to show that the gains of 
proportionate sampling may be obtained without stratified selection, 
by weighting the results m the separate strata so that in the combined 
sample result the various strata are properly represented 

(d) Although the gams from it may be small, proportionate 
stratified sampling ^is very widely used One reason is that it is a safe 
thing to do the precision cannot be worse than if the sample is drawn 
without stratification, and often it is better Secondly, it is an easy 
thing to do quite often, as in our factory example, it can be done with 
little or no effort Thirdly, because the sample js sclf-weighting, the 
calculations are simpler than with the use of either of the two methods 
mentioned in the preceding paragraph 

{e) The gams arising from stratification are often greater when the 
samp mg units are clusters than for the selection of elements Examples 
of stratified samples of clusters are discussed in Sections 18-21 

^ For a proportionate sample of elements, one should not waste 
much time considering the exact variables to use for stratification In 
most cases the researcher can choose readily the useful stratifying 
variables, in so far as they arc available In our example of the factory, 
ttie departments suggested themselves Perhaps sex, age, work sections, 
or job classification could have been used instead of the departments or 
as additional strata 


10 A Sjistematic Sample 

a **'' '“Ployees of a factory (students of 

vtSs wouldT “ Proporttonate Zpk of tnd. 

the pavroll has h ^ \ sample design Let us assume that 

the nam '" ^ and that wtth.n the depart- 

XhabenLlirw “ haphacard order or perhaps 

400 - TS ~ , , ^0 ? "■“* ™ ''"■y ™PW=e to have a 

to designate the numbers a. - a(w![/^,rt ^ °b 

t »/iV) that were to be selected in each 
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department, then from the Nk employees in the department ns random 

numbers were selected * ^ 

In practice a simpler device is commonly used systematic 

sampling This is probably the most widely ' 

of taking every itth individual after a random start from 1 to k In thi 
instance one luld take a number from 1 to 30 from a table of random 

W,,S ,h., -b., - • " » 1. ""S. 

If the random start was number 28, we snouio iia 
28, 58, 88, 118 11,998 in the sample These numbers rrfer 

consecutive numbers of the employees as they appear on the list ihe 
estimate of the mean here, as in a proportionate sample, is the usual 

simple mean of the sample However, the estimation of 

the variance of the mean is not clean cut (1, PP 199 , , 

If the payroll cards of each department had been shuffled thor- 
oughly tfor? they were ordered on the list the ^ 

would be equivalent to a proportionate stratified ra „ocedure 

the latter, the shuffling process is not nece^ary the p oced„^^ 

of as separate independent choices in each 
equivalent of a shuffling process With the 
tematic selection this shuffling is lacking However, ^ 
instances the haphazard arrangements are 

similar to random selection within the strata, in those cases the formula 
of stratified sampling would be used 

est var (J/) = SicJ 

Nevertheless, we should be wary In afat'fi-=d random selection 
the arrangements of the units withm the strata may h<=-B"°^ because 
the randL selection will provide the necessary 'hufflmg With ^ 
tematic sampling there is need for reasonable — 
arrangements of the sampling units within strata may h= r=sa ded a 
If they were random There are schemes for using seieral differenl 

random starts rather than just one . from 

The researcher should be alert for two kinds of ^ P=> 
randomness m the arrangement of the population ^ 

these situations CMS.s, or ma, eent. a systematic sample should 
avoided or modified 
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(fl) A trend Imagine that, unknmvn to tis, somebody sorted all the 
3000 employees m the foundry according to increasing senionty, and 
that IS the order in which they appear on the list from which we shall 
select the sample Then we are to select our systematic sample of every 
30th after a random start One of the 30 poss ble samples would select 
the employees numbered 1, 31, 61 2971, another sample would 

consist of numbers 30, 60, 90 3000 In the latter possible sample 

each employee has more senionty than his counterpart m the former 
by 29 ranks The means of these two samples would be widely divergent 
as regards seniority and other vanables strongly associated with it 
Hence, there is a great deal of vanability among the 30 possible 
samples, results on any characteristic strongly related to senionty 
would be dependent on which of the 30 possible samples happened to 
be chosen 


(A) A cyclical fluctuation Imagine that one of the departments is 
composed of work sections each of which contains 10 employees and 
that somebody has arranged the listing so that the 10 employees of 
each section are together and in the order of their senionty Thus we 
see a cyclical fluctuation of seniority with a “period” of 10 employees. 
Now if a sample of every 30th employee is selected, there are 30 
possible samples Three of these samples (those with the random starts 
in \ select the most senior employee m each group of 

10, three others the most junior, and the other samples would fluctuate 
simi ar y Again, we see a great deal of vanability depending on which 

among the 30 possible samples happens to be selected (1, pp 160-174) 

In addition to an unduly great amount of vanability, there is 
ano er scrips objection to a systematic sample under these cir- 
ums ances c wlculations made from the sample will not show this 
^ Thus the true standard error may be grossly 

underestimated from the sample data 

Nevertheless, systematic sampling is used very widely Because of 

for rf h H ‘‘^8'”'=''' Of » niauy kinds of populations, the formula 
or stratified random sampling is freqnenUy an adequate apprnsima- 
tion as we noted above However, suppose tLt wtthin each department 
, \ ^ ^ arrangement (by work sections, for 

sample) which makes for stratification within each department That 
stratification would not be reflected m the formula based on the 
assumption of simple random selecnon within the department A 
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formula which will be a useful approximation in these cases is one 
which is based on the successive squared differences among the n 
selected elements in their order of selection (1, p. 180; 22, p. 22}): 

est. var. {S„,t) = (1 — /) 2n (n - 1) ^ ~ 

In Sections 12, 19, 20, and 21, examples of systematic selections of 
clusters are given. In Section 18, a formula similar to the one above is 
discussed in application to a cluster selection. 


11. Allocation to Strata 

A method of using stratihcation to increase the 
sample mean is the deliberate use of different sampling rates in the 

various strata. The estimate J.' = 2 can be made most prec«e 

for a fixed cost if the sampling rate within each 
directly proportional to the standard deviation within the ^'catum and 
inversely proportional to the square root of t e . p j 

that stratum. Vt is, for a minimum standard error of , often called 

“optimum allocation,” make proportional to ^ 

Here J, is the cost per element, so that ZnsA is the fixed total cost 
related to the number of interviews (1» p- "73)- 

Several points may be noted: ^ m the 

(a) If the cost per interview is the same m eac . ’. 

problem becomes one of allocation of a fixed number (a) 
to the various strata. In that case the sampling ca<c >" 'ach stratu 
should be made proportional to standard deviation 

(i.) This “optimum allocation” of the sample may also be vies 
as that which yields a desired variance of the estimated mean for .he 

‘'“‘(To?na"nd"y rough estimates are usually available. How- 
• • ‘c not needed here; convenient rates which arc roughly 
c^er, precision ordinarily suffice. The difTerence in 

~n" U between an optimum allocation and another which 
is only roughly like it (3, p. )• 
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(d) For the reasonjust given, ordinarily it does not pay to resort to 
disproportionate allocation unless there are substantial differences 
among the estimates for or those for J\ among the various strata 
If thos-* differences are large, the gam over proportionate sampling 
may be large However, in the estimation of proportions, usually no 
great gam may be had through disproportionate sampling, their 
standard deviations are '\/^* (1 — />a), and that quantity is not sensitive 
to the kind of fluctuations one usually encounters with values of ph 
between 10 and 90 

(f) Disproportionate allocation should be used only with caution, 
perhaps only on expert advice The optimum allocation for one item 
on a survey may result in large losses of precision (greater standard 
errors) for some other items on the survey Furthermore, although the 
self weighted” proportionate sample makes for easy computations, 
the calculation of the sample mean of a disproportionate sample in 
volves weighting m inverse proportions to the sampling rates The 
weighted calculations of a disproportionate sample may be costly This 
added cost of tabulation, not included in the formulation of “optimum 
allocation above, should be considered before a sample design with 
disproportionate sampling rates is adopted 


CLUSTERING 


12 Clusler Sampling 

In a factory many instances would arise that would call for a 
c ustcr sample It might happen, for example, that employees would be 
sc ccte not individually but m clusters of work sections Let us move 
nr r° illustration Let us say that we want a sample 

or„h h w' r '2.000 dwellings of a city, with equal 

probability of se ectton for each dwelltng Were a list of the city’s 

fn 1 m P^^Oures of samphng discussed in Sections I, 

nm 'i^i’u, too Supposc, howcvcr, that such a list 

aval a e and it is deemed too costly to prepare Supposc, 
furthermore, that it is desired to economize on the costs of locating 
dwdhngs by means nf samphng entire blocks The ent.re area of the 
city s map ts divided into blocks along identifiable streets and other 
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natural boundaries so that ctcry dwelling is located clearly inside one 
of the blocks The blocks are numbered consecutively, this numbering 
establishes a list of the blocks, the numbers going from 1 to 750 The 
number of dwellings per block is variable, but the average is 12,000 - 
750 = 16 Now a sample is chosen by selecting in a random manner 25 
blocks out of 750 and including in the sample all dwelling units found 
within the boundaries of each of the 25 sample blocks Note 'ha"he 

probability of selection of any dwelling in the city IS 25/750 — / 

In this example the elements aic dwelling units, since the analysis 
will be in terms of characteristics of dwelling units The sampling unit, 
however, is the block, the selection is made from a complete list of al 
the sampling units In the population-that is, the numbered list ol 
blocks Each element belongs to one, and only one, of these sampling 
units The selection of each sampling unit results in the selection of the 


cluster of elements which it contains 

Cluster sampling is the name given to methods of selection in 
which the sampling unit, the unit of selection, contains more than one 
population element, the sampling unit is a cluster of elements In our 
illustration the block is the cluster, composed of dwelling units a 
elements But just what is a cluster or an element is only a matter o 
practical expediency In some studies the dwelling unit will be >'=garde 
as a cluster of persons, whereas in another study the population ele- 
ments might be blocks or even cities The same physical population 
the people of the United States, may be regarded in turn as compose 
of units which arc states or counties or cities, towns, and township , 
blocks or dwellings or, Bnally, the individual persons ^he element 
the population are defined m accord with study objectives e 
are defined in conformity with the requirements o a prac ica 
economical sampling design applied to the physical distribution of the 
population (3, pp 135-146) 

After the^Lmdual elements are defined, ^ 

whether they should also serve as sampling units or whether, c 
wise. It may be more economtcal and practical to define a sampl mg 
unit which IS a cluster of the previously defined elements 
be several examples given, in later sections, o situations w 
to the use of clusters The choice of clusters is generally “ ™°SnUi 
in the sampling procedures of some existing eatur« in i 
of the popu^lation Thus, one might study the employees of a factors 
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and select work grouDS, one might study the students of a university 
and select them m clusters of classes, one might select dwellings in 
clusters of blocks In any case, one must ensure that every element of 
the designated population belongs to one, and only one, of the clusters 
Otherivise, special measures must be taken 

CLUSTERS OF EQUAL SIZE Qusters of equal size rarely exist in 
society They come into being only as the result of planning Army 
units, the units of large housing developments, and some work sections 
m large establishments may sometimes be of equal size, or nearly so 
Sometimes, however, the sampler creates equal-sized clusters where 
none existed before, as in the example given below Furthermore, it is 
common practice to create equal sized subsample clusters by the 
procedures of sampling unequal sized clusters with probabilities pro- 
portional to size discussed in Section 20 Clusters of equal size can be 
treated simply as a special case of samples of unequal size However, 
the subject of clusters of unequal sizes is rather complex 

As an example, take the file of subscribers of a newspaper There 
are 12,000 subscribers served by earner routes There is a card for each 
subscriber in the file The 100 to 200 cards of each carrier route arc 
kept together, and neighboring routes follow one another An interview 
survey of about 400 subscribers is wanted, and in order to save travel 
time It IS decided to take clusters of 10 subsenbers each It is estimated 
that 10 short interviews m the same neighborhood can be generally 
obtained in half a day’s work on the spot 

Now, imagine that the file is divided into 1200 clusters of 10 
consecutive cards each A sample of 40 of the 1 200 clusters is to be 
selected The drawing of 40 dificrem random numbers from 1 to 1200 
would give a random selection of the clusters * However, our aim is to 
be practical, and in practice a systematic sample of clusters would 
generally be taken (sec Sections 10 and 18) Let m say that after a 
ran om start from 1 to 30, c\cry 30th cluster was chosen, we thus 
base 40 clusters of 10 cards each Altogether 400 out of 12,000 were 
selected each «tih a probabtl.ty of selection of 1 in 30 

o wide the file into 1200 clusters would take some time, so 


denote* 40 indtptndent selections with equal 
I«P«lauon The sample of 400 cases u 
I ^ choices Thw has important consequences for the 

sampling error, as ducussed in the next section 
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would the counting of 290 nonselccted cards for each cluster of 10 
selections. In practice, some approximate procedure might be used, 
such as the use of a ruler to measure off the clusters. Let us say that 
of the 400 subscribers interviewed 148 gave a “yes” answer to a 
question. The number of “yes” answers out of the 10 interviews in 
each of the 40 clusters were also obtained. These are, with the sections 
arranged in the same order in which they were selected: 

47113536232214463271 

55468203569210347524 

The sample mean is calculated as before by dividing the sample total 
by the number of cases: 


In the present instance we have 148 *= .37. 

Denote by m the number of clusters and by A the number of elements 
per cluster, so that mA n. (In the present instance we have 40 X 10 * 
400). Let Xx denote the sum of the values of the characteristic x for the 
A values in the ith cluster; in our example, the 40 values of A', are 
given above as 4, 7, 1 ... 5, 2, 4. The mean of the *th cluster will be 
denoted as x, = Xx. These 40 values may be found by dividing the 
values above by ^4 = 10; hence they will be 0.4, 0.7, 0.1, • • • 0.5, 0.2, 
0.4. These represent the mean of the characteristic for the elements 
in the cluster— that is, the proportion of subscribers in the different 
clusters who said “yes.” The subscript “ec” is used here to denote 
“equal clusters,” whereas the subscript “c” is used later when the 
clusters are not necessarily equal. It may be noted that the sample 
mean is also equal to the mean of the cluster means: 



In our example, ^ (-4 + .7 + .1 + • * ■ -f -2 -f .4) = .37. 

In order to obtain the sample mean, it is not necessary to go to the 
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trouble of calculating the 'icparate cluster values However, wc need 
them to obtain the estimated vanance of the sample mean 

est var (l^') = (1 - ii. 

m 

1 ^ 

Here S6^ = - ^ 2 (f, — the viriance of the cluster means 

around the sample mean Note the similarity of the formula to that 
for a simple random sample here, too, we have a variance divided 
by the number aj independent sampling units The units involved in the 
vanance calculations are also the sampling units the clusters The 
quantity (1 — /) is a»^ain the usually inconsequential “ correction for 
finite population ” ^Vith m clusters selected out of a total of M in the 
populatian.m/M.sthesamplmgfraction, and (I -J) = 1 -m/M In 
the present instance we have (1 — y) s= j _ 40/1200 *= 1 — 1 /30 
We have 1 

var 

967 X 001259 001217 


HcreS(i. ^ +(2- 37)> 

+ ( 4 - 37)' = 1 964 

The standard error .s V 001 ^ “ 035, and the conBdence .ntervals 

_f ^ \ "lake the statement that the proportion 

yes answers that wnnid have been obtained by a similar survey 

•More convenient calculalional fomn arc 

™ (>.) - (1 -/)v[— 77 (^ 4 ; -.O] 

This last form comes to — ! F 1 1 

30 4i> X 39 X 100 “ 40 

HcrcW-4. + 7i + f +2r + 4i=744 

39 too f 40 048) J *= 0504 "Itaj quantity, as above, is the value of 

.he esliinaie of the vanance of singfc dinten 



Selection of the Sample 207 


of the entire population of 12,000 subscribers is between 30 0 and 44 0 
percent That statement has a 95~percent chance of being correct and 
a 5-percent chance of being wrong It should be noted that this 
formula for the variance is entirely appropriate only if the m clusters 
are made with m random choices But for practical reasons the selection 
was systematic The results of our systematic selection would be 
equivalent to a random choice only if the order of the clusters m the 
file had been thoroughly randomized They were not, and we are 
told that “neighboring routes follow one another ” This matter will be 
treated in Section 18, where we shall find that for our example the 
approximation is not bad 

CLUSTERS OF UNEQUAL SIZES When the cluster is some existing 
human group, it will usually contain varying numbers of elements 
This will be true of dwellings in blocks, employees in work groups, or 
students in classes There are several important consequences The 
planning and administration of the study must reckon with the fact 
that It has no exact control over the size («) of the sample For example, 
the sample of 25 blocks, mentioned earlier m this section, will generally 
not contain exactly the planned 400 dwellings but will contain more or 
fewer, depending on the sample of blocks that happened to be selected 
Sometimes the variation in the size of the sample is reduced through 
the use of information on the sues of the individual clusters (see Sec- 
tion 20) Let us assume, however, that we are talking about a simpler 
model in which m clusters are taken with random choice out of the M 
clusters which compose the population The estimate of the mean is 
usually calculated as the simple mean of the sanjp)e cases x/ = 2 xjn 
The estimate of the mean mav be seriously biased if m is small and 
there is inequality m the size of the clusters (12, Chap 6) Thus, the 
sample mean is no longer equal to a simple mean of the cluster means, 
as It was for equal sized clusters However, a useful analogy may be 
found m that the sample mean is equal to a u.eighted mean of the cluster 
means, the weights are the relative sizes of the clusters Thus 

= — 2 W Here iV' — — *= — tlie average number of 
m jv' mm ^ 

elements m the sample clusters Hence NJN' is the size of the cluster 
relative to the average size in the sample 

The variance of this estimate has an analogous form to the 
variance shown before 
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es. var (./) = (1 _ /) ±[ ^ 5 - ./)=] 

That IS the squared deviations now arc multiplied by the squares of the 
relative weights of the clusters 


13 The Effects aj Clustering, Intraclass Correlation 

Of the vanous modifications of sample design (listed in Section 7), 
clustering usually has by far the greatest effect If we compare a sample 
of n independently selected elements with another sample containing a 
like number of elements but selected in m clusters, we note that the 
number of independent choices involved is reduced considerably 
Although the former sample would be well spread over the population, 
the latter would be bunched m spots In the example in Section 12, 
the sample of dwellings was confined to only 25 of the city’s 750 
blocks This clustering would be of no consequence if all the elements 
in the population were scattered at random into the different clusters 
In most practical cases, however, we find that the elements in a 
cluster tend to be more like other elements in the same cluster than 
like elements in other clusters The various dwellings in the same 
block will show a greater homogeneity than a similar number of 
dwellings scattered throughout the city The measure of this homo- 
geneity IS the intraclass correlation 

In the example of newspaper subscribers of Section 12, we found 
that the estimate based on 40 dusters of 10 subscribers each was 
subject to a variance which was estimated as 

est var (l„’) = (1 ^ 001217 

m 


The standard error was estimated at 001217 = 035 

Suppose that after this duster sample was selected the researcher 
mtstakenly used the formula for the standard error of a simple random 
sample, which is mappropnate for the sample design What ts the 
consequence of thts mistake’ In our example he would take (see Sec- 
tion 1) 
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•'id -/) 


p' (1 - p') ^ . 

n — 1 


.37 X .63 


= V.000565 = .024; 


also 1.95 X .024 = .047. 


Hence he makes the statement that the population value was within the 
limits of 37 percent ± 4.7 percent, and he would say that this statement 
has but a 5-percent chance of being incorrect. We are wiser; we know 
that the true standard error is .035. Hence 4.7 percent is equal not to 
1.96 standard errors but only to .047/.035 = 1.34 standard ermm. In 
the appropriate tables we find that the probability of a statement based 
on a confidence interval of 1.34 times the standard error has a prob- 
ability of 0.18 of being incorrect. Hence, the consequence of the re- 
searcher’s mistake is that his confidence, and that of his readers, in 
his results is misplaced: his statements will be incorrect in the long 

run not 5 times in 100 but 18 times in 100. ^ v •- * 

The effect of clustering on sample results is often senous. Yet it is 
disregarded very frequently. It is quite common to see reports ol 
estimates which grossly underestimate the actual variability of *e 
sample and consequently grossly exaggerate the significance of t 
sample results. Marks (17) found that for the revision of the Stanforf- 
Binet scale the ratio of the correct standard error of the mean I.U- 
to the incorrect s/v^is 3.36. Therefore, the res^mher using 1.96 
of his incorrect “standard errors” is using but 1.96/3.36 - 
the actual standard error. ^Vilh the use of the incorrect mtervals Ae 
researcher would be making incorrect statements not 5 times in luu, 
as he hoped, but 56 times in 100. 

A convenient measure of the effect of clustering ran be given in 
terms of the coefficient of intraelass correlation, rho. This actually i 
based on the ratio of two variances, and it can be estimated from two 
values we already have. The variance of our cluster sample is: ( J) 
hVm ” .001217. The value of (I -/) ^1" = -000565 is a usable esti- 
mate for the value of the variance to which a simple random sample 
of the same population would be subject. Their ratio is: 

"1-1- cat. rho (d — 1). 
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In our example, A (the number of elements m each cluster) is 10 
Hence we have 


1 + est rho HO — 1) 


000565 


= 2 15, 


and est rho = 


10 - 1 


+ 13 


That IS the variance of the cluster sample in this case is estimated as 
2 15 tim^s greater than the variance one would expect from a simple 
random sample of the same number of elements This ratio of the 
variances can be expressed as due to an estimated rho of + 13 

When rho is positive, the ratio of the variances is greater than 1 
and the cluster sample has a greater variance than a simple random 
sample of the same size would have The maximum possible value for 
rho is +1, in which case the ratio has a value of A This corresponds 
to a case of complete segregation of a characteristic all the individuals 
o every c uster are exactly alike with regard to that characteristic. 
Note,aso, that the effect of clustering is equal to rho (A - 1), hence, a 
relatively small rho may have a senous effect on the variance if the 
size 0 t e c uster (^) is large In the case of clusters of unequal sizes, 
the average size N' of the cluster can be used in place of ^ In a 
su samp mg esign (Section 19) the average number of subsampled 
elements per cluster plays the same role 

variance of a cluster sample is almost always greater than 
- 1 ° ^ ® random sample of the same size However, this is a 

selected'f n ^ *°g'cal necessity For example, if balls are 

takpx um,” It makes no difference whether one 

in the ^ the balls will be randomly distributed 

at o nf h a value of 1 for the 

Tot ente^^^^^^^^^ ""f. ^ ^ho However, people do 

tive rho is nos.; mixed ” Furthermore, although a nega- 

P I e, It IS a rare phenomenon for social variables The 
lowest possible value is — * 

^ j» corresponding to a zero value for the 

ratio 

j “> b<= positive That is, the 

individual, associated with human puups tend to resemble one 
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another These tendencies to homogeneity in groups hold for most 
characteristics The homogeneity of people— the segregation of char- 
actenstics— IS greater than would exist if people were selected into 
groups by random choice These tendencies may be due to selection, to 
mutual influence, to the correlation among characteristics, or to a 
combination of these In any case, the tendency is a social phenomenon 
in that rho is a measure that belongs to the group as such, it has no 
meaning for the individual except in so far as he is considered as a 
member of a group (or cluster) Rho is a measure of the fraction of the 
total vanance which elements m the same sampling unit, the same 
cluster, have in common It may be looked on as measuring the amount 
of homogeneity or segregation of the elements within a group of units 
Given a set of elements, the greater the segregation into separate units 
of similar elements, the greater is the homogeneity within the unit 
It should be of great utility in the social sciences as a measure for 
description and comparison Of course, rho is specific for a char- 
acteristic For the same grouping of individuals, different variables will 
exhibit different rhos Also, for any specific characteristic, the value of 
rho depends on the actual distnbution of the population into the 
groupings being considered 

That individuals are segregated in social groups is of importance 
to the sampler because he often uses those groups (or some approxima 
tion thereof) as sampling units in the process of selection Therefore, 
the variance of sample results will be influenced by the clusters used in 
the selection process We see, then, that the precision of the results of a 
cluster sample cannot be given simply in terms of the numbers of 
elements (cases) in the sample The numbers of the other units selected 
—1 < , the numbers of the different kinds of clusters used— will be of 
importance also 


PRACTICAL PROCLDURES 
14 JOcoromj 

Tlic results of a sample bised on clusters are almost alwa>s subject 
to greater sampling error than those of a sample of the same number 
of elements selected inclividualU ^\h^, then, use clustenng m selection’ 
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Because the cost per element in cluster sampling is less, often sub 
stantially so, than for a sample of individually selected elements 
Clustering should be preferred over individual selection m so far as the 
lowering of the cost per element due to clustering is greater than the 
increase in the variance per element 

Let us begin with the data of the 400 subscribers in 40 sample 
clusters m Sections 12 and 13 Suppose that the same charactcnstic 
^to be n-easured again next year and the cost of the study is specified 
e want to compare two sample designs in order to use the one which 
yields the smaller variance for the available expenditure First, estimate 
ow many sampling units can be obtained for the available money 
under each of the two designs Suppose that it is estimated that with 
one esign 0 subscribers in 30 clusters may be obtained, or one 
Quid get 100 subscribers selected individually Which of those two 
smaller variance^ The vanance of the cluster 

sample is 

approx var (xj) » ii! 

m 

of anv rnn IS neglected in this discussion, and it is seldom 

Imnle Section 1) We found in Sectmn 12 that m our 

0504/30 '= ^ sample of 30 clusters is estimated as 

The vanance of the nmple random ^m„le „ 

approx var (x,') = il 
n 

Again, we neglect the factor (1 - n j , u 

J) oincc we deal with a proportion 

234 


= f'P' (1 - p') 


1 400 ( 37) ( 63) 
399 


This IS a usable estimate nf fia,« 

formula was given in SectiA i '"^^nce of individual elements (The 
tions 12 and 13 ) Therefore 

234 ’ of a sample of 100 elements 1 .' 

estimated as ~ = nm-jd c - 


— a sample oi luu eiemcms w 
100 00234 Since for the available money the cluster 

sample yields the smaller vanance the u .. k 

preferred greater precision, it should be 
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Alternately, one may begin with a fixed precision that he wants 
the sample to have Suppose, now, that our purpose is again to measure 
the same characteristic on another occasion but now it is specified that 
a precision of 6 percent is required at the 0 95 probability level This 
means that a standard error of 3 percent = 03 is required That is, 
the variance of the sample mean is to be ( 03)^ = 0009 Which of the 
two designs will yield a sample of that precision for Jess cost^ Using 
the equation for the vanance of a cluster sample, we find the number of 
clusters necessary for the desired variance of 0009 


m for variance of 0009 = 


0504 

0009 


= 56 clusters 


Using the equation for the vanance of a simple random sample of 
elements, we find similarly that n for variance of 0009 ** 234 / 0009 = 
260 elements ^ 

Now the cost of visiting 560 dwellings m 56 clusters is to be esti 
mated, also, the cost of 260 dwellings selected individually Of these 
two designs, either of which would vield the same \anance, we choose 
that which costs less Note that the required precision dictates a sample 
size, m numbers of elements or clusters, and not a sampling rate tor a 
desired precision and for a given variability among the sampling units 
of the population, the required sample size is about the same for a 
small as for a large population 

Two views of economy have been presented for a fixed variance 
we sought the less costly of two designs, then for a fixed cost we sought 
the design with the smaller variance In both cases we sought the 
design which yields the least variance (ir, the most precision the 
most information) per unit cost The superlative in the last sentence 
IS justified because the search for economy is not limited usually to two 
alternatives only, the principles and procedures discussed hold for 

takes 2 15 as many elcmcnu in the cluster tample as in the samph 

of indmdusU to obtain the same vanance Thu n the same ratio as we found in 
Section 13 for the ratio of the vananccs of the two designs with equal numben of 
elements That fraction u determined by the »i*e (j 4) of the cluster and the intraclass 
correlation (rho) for the characterutie in the speciHc clustering 
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the comparison of anv number of altemali\e designs For example, in 
the case above we might investigate whether a cluster of 3 or 5 or 20 
elements might be superior to both single elements and to clusters of 10 
If the cost per element were the same for the clusters of 10 elements 
as for the individual elements, the latter would usually show up better 
It will generally yield the same vanance for smaller n or the smaller 
variance for the same n However, the cost per element is generally 
not the same for elements in clusters as for elements selected indi* 
viduallv Hence, the comparison should be made between variances 
per unit cost 


As these comparisons are made, it becomes obvious that the 
number of elements-say, interviews-in the sample is not the sole 
important factor as regards cither the cost or the vanance of sample 
esigns e numbers of all other sampling units selected— such as 
dwellings blocks, towns, cities, couniies-are also relevant Furiher- 
aspects of design— stratification, varying probabilities of 
ion, an met ods of estimation— should be considered also 
^ r studies those aspects are not so important as 

study ° ^ '^stenng, either for the vanance or for the costs of the 


questions for which vve do not have definite 

answers are pertinent 

w needed csiimaics of variances and of 

similar Mirv ^ ^i " them on the basis of past expcnencc with 

of cost fartn expert or conduct a pilot study Good estimates 

of CO t factors are especially difficult to get 

should detrr ^od the precision needed’ The user of the statistic 

sar^rr^X: r.h'e" >'V. 4h. .ay .ha. a 

warran.ed hv c ®*^*“*'*^ '* yields are more precise ihan IS 

.hem » a„ elrc""'* 
rational picture, the desire^n?" 

samole size and iKe precision would always determine the 

.hatd=.erm’.na..on AcX'^«X''omfi d" 
well defi„ed-or the mailable X ° flmd 
(e) For studies with several n- 

measure, how do we evaluate tZ ‘"^P^^^nt characteristic to 

the relative importance of each’ How 
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do ^^c arbitrate between the conflicting nnswers on the desirable size 
and design of the simple’ This area of decisions is c\en more obscure 
than the preceding one 

Despite these difficulties, theory docs provide general outlines for 
the decision on the required sample size and it can be of great help in 
choosing an efficient design Fortunately, moderate departures from 
optimum design do not incur heavy losses in economy Hence, rough 
guesses may serve in place of better estimates, and useful compromises 
between conflicting aims in design can often be made 

The designing of samples is mainly an engineering job, the avail- 
able theory and the knowledge of results with similar materials should 
be utilized to produce a desired result with the available resources 
and With the greatest economy This, at any rate, should be the goal, 
but the use of the superlative in the preceding sentence is immodest—it 
usually represents a level of aspiration rather than of achievement In 
making economy the aim, we should understand that cost is to be 
understood broadly to mean effort m general Since effort as well as 
money available for research is limited, economy helps to increase the 
total quantity and quality of research output (1, p 50fi) 

15 Practicality 

A probability sample cannot be created by assumption, nor will it 
be “given,” as in the examples of elementary statistics The dictum of 
“quota” samplers to their interviewers, “Go out and get a random 
sample,” is most impractical The interview cr is not capable of doing it, 
nor IS his dispatcher The need for a mechanical method of selection 
has been stated in Sections 4 and 5 Now we want to emphasize the 
need for taking care to translate the theoretical model of selection into 
a complete set of simple, practical instructions 

It IS necessary to give the field interviewer simple and clear in- 
structions for the carrying out of his tasks The less attention the 
sampling instructions demand of him, the xrore he can devote to his 
pnncipal and difficult task of interviewing For example, m order to 
identify a sample segment the interviewer should not be asked to locate 
a long arbitrary straight line marked on a map—but he can locate a 
street His sampling instructions should be confined to locating streets 
and addresses, listing occupants of the household, and so on These 
tasks are difficult enough in some cases 
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Of course, “clear, simple, practical and cumpUte instructions” 
represent aims rather than fulfillment Actually, in the fitting of 
practical field sampling procedures of large-scale samples to the 
statistical model of the design, there will often be some unfilled gaps 
see ection 1 6) Part of the art of practical work is m guessing what 
irregularities, where and how much, one can afford to tolerate 

The sample design is no better than the weakest link in the entire 
procedure Each sample design is an adaptation of sampling theory to 

‘he distrihufion of the 
of the fiew’r*' facilities for communication, the nature and training 
inclndf. th researchers engaged m the task They also 

research administration as well as of the users of 

bg^cal mot"’' ‘■^ccptiviti to. and understanding of, the methodo- 
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practice Thus the population value obtained in a census by means of a 
single observation on each individual is subject to error Because of 
the theoretical fluctuation due to the variable-response error, we may 
think of a distribution of possible population values The mean of this 
theoretical distribution of population values is the “expected popula* 
tion value ” The difference between this value and the “true value” of 
the parameter is the bias (1, pp 292-317, 11, pp 147-154) 

In addition to the nonsamphng errors of bias and of variable 
response, sample studies are also subject to sampling errors In the 
general study design, all of these errors may be considered together as 
constituting the total error of the sample This total error is the square 
root of the sums of squares of two quantities The first is the standard 
error of the sampling distribution, the sampling error The second 
quantity is the combined effect of the two kinds of nonsampling errors 
which we called variable-response error and bias That is (total error)^ 
= (standard error of sampling)* + (non*samp]ing errors)* This rela- 
tionship may be illustrated by means of the three sides of a right 
triangle 



STANDARD 
ERROR OF 
SAMPLING 


The total error depends on the length of both of the legs and cannot be 
shorter than either of them The standard error leg can be shortened 
sometimes by a change m sample design, and always by taking more 
sampling units— either more clusters or more individuals Of the non- 
sampling errors, the vanablc-response errors may be reduced cither by 
taking more of something— observations per individual, or individuals, 
or interviewers- or by improving the precision of the methods of 
observation But the length of the nonsampling leg may be due mostly 
to bias, which can be reduced onl> through better survc> procedures 
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through improving the questionnaire, or the field work, or the coding 
and processing, etc * 

It may be ^\asteful to spend much money on a large sample in 

order to reduce the standard error if the nonsampling errors are 

allowed to remain large and vice versa In the general study design, 

the nonsampling errors should be considered together with the sampling 

error because together they constitute the total error of the survey 

An important special class of nonsamphng errors in social studies is 

composed of the errors of nonresponse These arise whenever a member 

of the population designated for the sample is not included in the 

results because his answer was “not ascertained,” because of total 
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VARIOUS STATISTICS 
17 About Various Statistics 

The discussions in this chapter were built around problems of 
estimating the mean of a populadon The restriction was for the sake 
of simplicity and convenience The choice of the mean rests on its 
basic importance The estimation of totals for the population is closely 
related to that of means (as discussed m Section 1) Furthermore, 
proportions are but a special case of the mean, and a great deal of 
social research is reported in terms of proportions, of frequency distribu- 
tions in classes The distributions may be of attributes, behaviors, 
attitudes, or opinions, they very often are expressed as proportions of 
the total 

In addition to the mean for the entire population total, the estima- 
tion of means for subdivisions of the total population is often of great 
importance These have been called “domains*’ “Any subdivision 
about which the enquiry is planned to supply numerical information 
of known precision may be termed a domain of study” (21, p 5) In 
general, the principles discussed in this chapter in terms of means of 
the total population apply as well to the means of the domains Hence, 
the results based on cross-tabulations present no new problems m 
principle However, it is true that, compared with the total population, 
in dealing with a domain one gets more often into “cells” so small 
that the problems of small samples, particularly the questions of non- 
normality, become important Hence, the reference to available tests 
for “distnbution-frec” estimates is particularly relevant here (sec Chap 
12) Furthermore, the effect of clustering becomes less drastic for 
many of the domains than for the enure sample because of the smaller 
number of elements per cluster 

Sometimes a researcher will say that the question of snmphng is of 
no interest to him because he wants not to estimate quantities but 
merely to measure relations This view may overlook the fact that the 
relationships arc measured in terms of statistics in comparisons of 
proportions, in correlation cocnicicnts, etc These staiisucs, too, depend 
on the individuals included in the sample 1\hcn some rclaiionship is 
expressed m terms of a number based on sample data, that number ii a 
statistic, a sample estimate of a population characteristic The statistic 
IS subject to sampling error, and the sampling error can l>c cxpressetl in 
terms of .a confidence interval 
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through improving the questionnaire, or the field vvorh, or the coding 
and processing, etc 5 

It may be ^\asteful to spend much money on a large sample in 
order to reduce the standard error if the nonsamphng errors are 
allowed to remain large, and \ice-vcrsa In the general study design, 
the nonsamphng errors should be considered together with the sampling 
error, because together they constitute the total error of the survey 
An important special class of nonsamphng errors in social studies is 
composed of the errors of nonresponse These arise whenever a member 
of the population designated for the sample is not included in the 
rcsults-bccause his answer was “not ascertained,” because of total 
refusal of the interview, because of not being at home, because of 
illness or for similar reasons This is a nonsamphng error, it can occur 
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VARIOUS STATISTICS 

17 About Various Statistics 

The discussions in this chapter were built around problems of 
estimating the mean of a population The restriction was for the sake 
of simplicity and convenience The choice of the mean rests on its 
basic importance The estimation of totals for the population is closely 
related to that of means (as discussed in Section 1) Furthermore, 
proportions are but a special case of the mean, and a great deal of 
social research is reported in terms of proportions, of frequency distribu- 
tions in classes The distributions may be of attributes, behaviors, 
attitudes, or opinions, they very often are expressed as proportions of 
the total 

In addition to the mean for the entire population total, the estima- 
tion of means for subdivisions of the total population is often of great 
importance These have been called “domains’* “Any subdivision 
about which the enquiry is planned to supply numerical information 
of known precision may be termed a domain of study” (21, p 5) In 
general, the principles discussed in this chapter in terms of means of 
the total population apply as well to the means of the domains Hence, 
the results based on cross-tabulations present no new problems in 
principle However, it is true that, compared with the total population, 
in dealing with a domain one gets more often into “cells” so small 
that the problems of small samples, particularly the questions of non- 
normahty, become important Hence, the reference to available tests 
for “distnbution-free” estimates is particularly relet ant here (see Chap 
12) Furthermore, the effect of clustering becomes less drastic for 
many of the domains than for the entire sample because of the smaller 
number of elements per cluster 

Sometimes a researcher will say that the question of sampling is of 
no interest to him because he wants nor to estimate quantities but 
merely to measure relations This \iew may o\erlook the fact that the 
relationships are measured in terms of statistics in comparisons of 
proportions, in correlation coefficients, etc These statistics, too, depend 
on the indiNiduals included m the sample When some relationship is 
expressed m terms of a number based on sample data, that number u a 
statistic, a sample estimate of a population characteristic The statistic 
IS subject to sampling error, and the sampling error can be expressed in 
terms of a confidence interval 
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“Tests of significance” of sample results are basic in research The 
statistical tools required for many tests of significance are the same as 
those which are necessary for the construction of the analogous con- 
fidence intervals * Suppose, for example, that it is desired to make a 
test of significance between two means, each pertaining to a domain of 
the study The statistic used m the test is a ratio, the numerator is the 
difference of the two means, and the denominator is the standard 
error of that difference The variance of the difference for the case 
of two independent simple random samples is simply the sum of the 
variances of the two sample means For more complex designs, another 
term has to be considered— the covanance of the two sample means 
brought about by the design 

A frequently used test of the presence of relationship is the chi- 
square test based on the 2X2 cells of two dichotomies In the case of 
simple random samples, that test is very similar to the test of the 
difference of two proportions (20, p 203) In case of a more complex 
esign, t e chi-square test is not valid because its underlying assump- 
tions of the independent selection of sample cases is violated by the 
samp e design But the test of the difference of the two proportions 
may e made This requires that the correlations due to clusterings 
and to the other complexities of the design be considered 


COMPLEX SAMPLE DESIGNS 
18 Stratijied Cluster Sampling 
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systematic selection ^vould be equivalent to a random choice It was 
stated specifically, however, that neighboring routes were filed next to 
one another Therefore, the similanties, slight or great, that might 
exist in neighboring routes would be reflected in a selection of clusters 
rather evenly spread over the diflTerent neighborhoods of the area We 
shall assume that the chief effect of this systematic selection of the 
clusters is to yield a stratified sample of them (see Section 10) 

Each selection came from its own “implicit stratum” of 30 clusters 
(the interval was 1 in 30) However, we shall use an approximation 
called the method of “collapsed strata” and assume that each successive 
set of four selections was selected by random choice from a group of 
120 clusters Thus, we shall have 10 strata with the equal weight of 
1/10 for each There are four equal-sized clusters selcwted from each 
stratum The sample mean is calculated as before, but the sample 
design now assumed calls for a different formula for obtaining the 
estimated variance of the sample mean We need here a specific 
application of the general formula for stratified samples (Section 8) 

ft 

est var fest var 

The quantity within the brackets is the variance within one of the 
strata Within each stratum we have a sample of four equal clusters 
out of 120 Hence (as in Section 12) we have in the Ath stratum 


est var = [^(l - ^) T T ^ ~ ‘ 

Then, for the entire sample, we have 
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“Tests of significance” of sample results are basic in research The 
statistical tools required for many tests of significance are the same as 
those which are necessary for the construction of the analogous con 
fidence intervals * Suppose, for example, that it is desired to make a 
test of significance between two means, each pertaining to a domain of 
the study The statistic used in the test is a ratio, the numerator is the 
difference of the two means, and the denominator is the standard 
error of that difference The variance of the difference for the case 
of two independent simple random samples is simply the sum of the 
variances of the two sample means For more complex designs, another 
term has to be considered— the covariance of the two sample means 
brought about by the design 

A frequently used test of the presence of relationship is the chi- 
square test based on the 2X2 cells of two dichotomies In the case of 
simple random samples, that test is very similar to the test of the 
difference of two proportions (20, p 203) In case of a more complex 
design, the chi square test is not valid because its underlying assump- 
tions of the independent selection of sample cases is violated by the 
sample design But the test of the difference of the two proportions 
may be made This requires that the correlations due to clusterings 
and to the other complexities of the design be considered 


COMPLEX SAMPLE DESIGNS 


18 Stratijied Clusttr Sampling 


In Section 12 we dealt with a sample of subscribers as an example 
of cluster sampling The estimated vanance of the sample mean was 
given as (1 — J) s^/m That formula is applicable to the case where 
the sample of m out of M clusters represents m independent random 
selections It is basic to the more complex designs However, the 
simple random choice of clusters is not used frequently m practice 
btratification is used generally Very frequently the selection is sys 
tematic, as in our example of the cluster of subscribers If the listing 
of the clusters had been random-as after a thorough shuffling-the 
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systematic selection would be equivalent to a random choice It was 
stated specifically, however, that neighboring routes were filed next to 
one another Therefore, the similarities, slight or great, that might 
exist m neighboring routes would be reflected in a selection of clusters 
rather evenly spread over the different neighborhoods of the area We 
shall assume that the chief effect of this systematic selection of the 
clusters is to yield a stratified sample of them (see Section 10) 

Each selection came from its own “implicit stratum” of 30 clusters 
(the interval was 1 in 30) However, we shall use an approximation 
called the method of “collapsed strata” and assume that each successive 
set of four selections was selected by random choice from a group of 
120 clusters Thus, we shall have 10 strata with the equal weight of 
1/10 for each There are four equal sized clusters selected from each 
stratum The sample mean is calculated as before, but the sample 
design now assumed calls for a different formula for obtaining the 
estimated variance of the sample mean We need here a specific 
application of the general formula for stratified samples (Section 8) 

R 

est var (?«,') = [est var (5/)] 

The quantity within the brackets is the variance within one of the 
strata Within each stratum we have a sample of four equal clusters 
out of 120 Hence (as in Section 12) we have in the hth stratum 


est var (5«:')h = 30) 4 3 ^ 

Then, for the entire sample, we have 
est var (*„„')= 2 
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There are ten terms, of whicli the first is 

[S (J, - = ( 4 - 325)’ + ( 7 - 325)’ + ( 1 - 325)’ 

+ t 1 - 325)' = 2475 

Here (*„ )i = i ( 4 + 7 + 1 + 1) = (1 3) = 325 


Thesiimofthetenterms(of»hichthefirstii 2475) equals! 505 Hence 


est var (a-, 


.) = 967 1 505 = 00121 


In our case this approximation did not produce a variance smaller 
than the 001217 obtained without considering any strata in Section 12 
We might have used 20 strata of two clusters each Then the 
variance would come from the 20 differences between pairs of clusters 
ormu a used in systematic sampling, and mentioned in Section 10, 
utilizes all of the 39 successive differences That is 


est var (V.r.i) 


V 30y L40 2 39 
^ _1_ _1_ 1 1 » 

' 30 40 T 39 Tm ^ 


2 (^. - *en)’ 


A-«.i)’ ■ 


- 344 = 00107 


Ts'- 2)-^ U : 344~ +(!-•)■+ + 

ytelV appre^ry differet *''' 

, the cquivalem term 
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In Sections 19, 20, and 21, the selection of subsamples is discussed. 
The examples discussed in each of these sections deal with samples to 

n 

which the calculation of the simple mean ^x/n is proper. The calcula- 
tion of variances for subsamples in general will not be discussed in this 
chapter. However, if the subsamples in all the clusters are of equal 
size (or very nearly so), the formulas for the variances of the mean are 
similar to those given just above. Equal-size subsamples are obtained 
with systematic selection with probabilities proportional to size. An 
example from Section 20 is the subsample of 10 employees each from a 
sample of 40 work groups. A satisfactory approximation might be: 


est. var. (Jc,../) 


2m (m — 1) 


2 (Jc/ - 


2 X 40 X 39 




The use of 5,' rather than je, denotes the fact that, since we have only a 
sample of the employees of any section, we deal not with the exact 
value of S{ but only with its estimate x/. The subscript e-n denotes a 
sample composed of subsamples of equal size. 

No formulas for the variance will be given in this chapter for 
stratified samples based on dusters or on duster subsamples which are 
of different sizes. The possibilities for varieties of procedures both of 
selection and of estimation are very great. Differences which may 
appear minor to the unpracticed eye may have important effects 
on the design. It is wise to consult a sampling statistician with the 
design during the planning stage (1, pp. 234-267). 


19. Subsampling. A Two-stage Sample 

Suppose that a sample is to be taken of a city’s dwellings and that 
two designs are compared, with the aim of making a choice between 
them. One calls for 400 dwellings to be selected individually from the 
listing of the city’s total of 12,000; the second calls for a duster sample 
of 25 of the city’s 750 blocks (as described in Section 12). In each 
case wc have a sampling fraction of 1 in 30, a probability of 1 in 30 
for every dwelling in the city. Wc arc comparing two samples with the 
same expected number of dements. The objection to the sample of 
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individual dwellings is that its cost per element is high There are an 
average of 16 dwellings per block, hence it takes on the average almost 
two blocks to produce a sample dwelling The cost of locating the 
dwellings may not be serious within this moderate-sized city, although 
tn other cases of clustering the cost of travel may be of the greatest 
importance (see Section 21) However, no satisfactory listing of the 
dwellings exists Thus 29 nonsampic dwellings would need to be listed 
for every sample dwelling and this additional burden increases the 
cost per element appreciably 
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mrS 11 ® ‘‘“'••■''g* pcc block means that the 

fl6 tn t '^^”^oce of this sample to the first design is [I rho 
rho w n 'ir u‘“ -f foe a chalacteristic 

Rrcaterth* ^ r cluster design will be 2 5 times 

design the sa "l ° c”' ‘*“’8" The trouble is that with the cluster 

of the samr.s blocks tn the city The spread 

01 me sample ts very restricted ^ 

the cost increase in the size of the cluster 

co7pro™rh l-f) We may then look for a 

spread our samolc'mtn factors on the one hand, we wish to 

m the samole the d "'any clusters as possible m order to include 

we wnrX; 'do™ Thetr“ ■" ’*■= 

In other words we w h incurred by covering many clusters 

good compromise bet'* size which yields a 

That IS, after companrvWt,' ' ““•'‘obng effects of clustering 

other dcsigns-in this cam for m'”"""’' ''“‘Sns, we look at 

clusters However th(» e ^nipromise in the form of smaller 
venient to split them ° clusters are often such that it is incon- 
boundanes, so that thci ^ "^ost city blocks have streets for 

instructions but a ^rcas can be located with relatively simple 

Suppo;=that.mte‘d'’ot1b“°'\''"^'^“ 

e sample of entire blocks we investigate 
^Comparison is in the ma j 

« in terms of a constant number'^nf -i Section 14 The present discussion 

more rational grounds of a fixed translated into the 
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a third design that would triple the number of clusters in the sample 
It would take 3 X 25 = 75 blocks, and an average of 16/3 = 53 
dwellings in each The intraclass correlation does not change, since 
It is a property of the actual block but its effect becomes less than one 
third as great as before In a calculation of the effect of clustering, the 
size of the average subsample taken from the cluster appears as the 
multiplier for the case of rho = 01, vve have [1 4" 0 1 (5 3 — 1)] = 
1 43 as the ratio of increase of variance over the first design But this 
IS considerably less than the ratio of 2 5 of the second design over the 
first On the other hand, the cost of the third design is slightly greater 
than that of the second (75 blocks jn contrast ivith 25) but perhaps 
in this case considerably less than that of the first 

Suppose that after examination of a few other designs (say, 
clusters of 2 and clusters of 8), the third design, of clusters of 5 3 
dwellings, is adopted The plan is then to select 1/10 of the blocks 
and to select 1/3 of the dwellings within the selected blocks The 
probability of selection of any dwelling unit is 1 /lO X 1 /3 = 1 /30 as 
before Note that the inclusion m the sample of any dwelling depends 
on Its being selected m two separate events There are two stages of 
selection first a sample of blocks is selected, then from the list of 
dwellings found withm the sample blocks a sample of dwellings is 
subsclected This is an example of multistage sampling in general (see 
Section 21) (I, pp 215-267, 3, pp I35ff and 372-397) 

At each stage of sampling there must exist a listing procedure for 
all the sampling units among which the selection will be made (Sec- 
tion 5) The listing of the blocks of the city is accomplished by the 
division of the map into blocks and the numbering of these blocks 
After the sample blocks are selected, a listing must be made only of 
the dwellings in the sample blocks (15) The procedures of selection 
in the two stages arc determined separately, and each of them may 
be a simple random selection or stratified, systematic, etc In the 
selections of blocks and dwellings, as in many other undertakings, the 
use of systematic samples is quite usual In our example the blocks 
would be selected with the interval 10 after a start with a random 
number from 1 to 10 In each sample block an interval of 3 would be 
applied to the listed dwellings after a random start from 1 to 3 Then 
we would obtain in each sample block a number of sample dwellings 
which IS closely proportional to the total number of dwellings listed 
in the entire block m the proportion of 1 in 3 However, there would 
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be divergences of 1/3 and 2/3 of a dwelling from the exact frac 
tional ratio 

The numbering of the blocks may be used to introduce stratifica 
tion All that is necessary is to have rchtnely similar blocb (by 
geography or other variables) numbered consecutively The interval 
applied to that list will select one blocb from each group of blocks 
cquahn number to the mterval-in this illustration, one from every ten 
e design has general applicability similar considerations in 
ot er situations may lead to the same design of subsampling For 
examp e, we might use an interval to select among all the work groups 
ot a factory, subsampimg with another interval for employees within 
tne work crrmine “ ' 


20 wUh P,cbab,l„„s P,opo,t,md to Skc 

samnle t described for a two stage 

ThenrobLiInv f * ’''“'"S”'""’'" «>'eted sample blocks 
= 1/30 The '•""""’S >" 'he city was 1/10X1/3 

constant interval vtithn °he^hr''v ® '‘"’'"'"S' “ 

number of sample dlvell,„J *’ 1 °'^ 3'''“ >" each sample block a 

number of dwelliner«- 'vhich is closely proportional to the total 
will vary from hWk 'u ^ number of dwellings 
hand Z 'rons - Sreat deal On the other 

the number of samnlf. a convenience, we would like to keep 

we to accomplish th nearly constant However, were 

Within the larger blnrt increasing the sampling interval 

of selection of the d ii "^°nld thereby decrease the probability 
violate our stated 3 withm these larger blocks and thereby 

dwelling units m thj city P-'^'^ability (of 1 m 30) for all the 

if we increase the n V? ^Iso accomplish our purpose 

ratio that we decreasT \ selection of any block in the same 

block Thus in the n ^ ^ Probability of selection within that same 

“ have about .wiceThTulr”'^”*' a block which appears 
secutive block unit nu' h™”*' ”“™her of dwellings is given two con 
doubled from l/fQ thereby its chance of being selected is 

sample the interval of *hcse blocks falls into the 

sampling within it is doubled that is every 
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sixth address is selected after a random start between 1 and 6 Hence, 
the probability of selection of a dw elling in that block is 2/10 X 1/6 = 
1/30, as required (3, pp 393-397) 

The over-ail probability of selection of an> dwelling in any block 
IS the product of the two probabilities the probability of selection of 
the block, and the probability of selection of the sample dwelling within 
the sample block During the process of numbering the blocks we 
assign block unit numbers as “measures of size” (Zt) in accord with 
the number of apparent dwelling units m the block, and this measure 
Zi IS used in opposite and balancing wa>s in the two intervals pertain- 
ing to the two stages of selection It is to the list of consecutive block 
unit numbers that the interval 10 is applied The probabilities of 
selection are now represented for the two stages as 

10 3Z{ 30 

Here 1/30 stands for the over-all probability of selection of anv dwelling 
in any block in the city The probability of selection wiinin a specific 
(the ith) block is represented by l/3Zi, that is, by an interval of 3Z,. 
The measure of size Zt vanes from block to block, for most blocks it is 
one, for some it is nvo, and for others three or more 

The probability of selection is determined by the application of 
the sampling intervals and is independent of any inaccuracies and 
deficiencies in the measure of size (Zi) In so far as the measures of 
size are made proportional to the total number of dwellings in the 
block, the procedure yields equal numbers of subsampled dwellings 
from blocks which vary in total size In practice this process cannot 
be complete, and it need not be The subsample will vary somewhat, 
but It IS enough if most of the variation can thus be eliminated (15) 
Where does one obtain the information he needs on number of 
dwelling units in each of the blocks of the city so that he ma\ assign 
measures of size’ There are several sources of materials, one or more of 
which may be a\ailable for the city Some of these are census block 
statistics, aenal photographs, the city engineer or the city planning 
office, the real estate board or the chamber of commerce, the local 
bank, the local newspaper, some local public utility, etc Other 
sampling materials, as well as advice, may in some cases be obtained 
from the Census Bureau If all these fail, one may resort to some cheap 
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rough estimates (perhaps just a glance from a moving car) to he made 
for all the blocks of the city or for a large sample of them 

The use of block units (Z,) is aimed at equalizing roughly the 
number of dwellings subsampled from blocks which contain unequal 
numbers of dwelling units This procedure may be looked upon as an 
example of selection with “probabilities proportional to a measure of 
size Let P, denote the estimate of the number of dwelling units in 
the block And let us say that we want to obtain an average of 5 of the 
(estimated) dwellings per block Then we can take for our two stages 
of selection the probabilities of P./150 X S/P. = 1/30 A procedure 
for selecting the blocks may be the following (n) Assign measures of 
SIK P. to each of the blocks (6) Arrange the blocks in some desirable 
order, stratification may be obtained from the systematic selection 
because a block will be chosen from each successive sum of 150 of the 

f.s (r) After a random start from 1 to 150, apply the interval of 150 

0 obtain the selection numbers (d) Cumulate the measures of size P„ 

etTvV ‘L order This cumulation can be done 

the r" ''•''""or the addition of a block causes 

elected '’T T f ‘hat block is 

beint chosen ^ chance of 

the listinv^nf selecting dwellings within the block involves 

s o2d"m,h V fc"®* ■" “““ Thrn the interval of P./5 

Khat n ord r “orr from 1 to P./5 

1 m ^s lrvl '"“■"'“-or- of an interval less than 

Some 3al m "'‘“h'r smaller than 5 for P. 

maTbrou ir, ? “■0 smaller blocks they 

Uoefa or thl mav '‘?T- " he attached to the largeV 

On thL other hand\?T “ larger number arbitrarily 

selected into the s ? ' ‘“S' “ Breater than 150 may be 

Sen ^ tJ.ee, 

,0 p./io selecting dweUings within it should be halved 

all vhjjrbrsibtnXoirnX” " r 

one thing, the number of dJcllinffl^m fr™ the various blocks For 
an exact multiple of the sampling .„t^“^h'’ f J’ 
sample dwellings actually obminri mijtrfcer’th iJT T 
of one dwelling) from that expected on rt, ‘‘‘‘f'rent (by the fraction 
kc on the average Another source of 
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vanation is more important it may be summarized as arising from 
the measures of size not being exactly proportional to the actual 
number of dwelling units listed in each of the blocks This divergence 
may be due to a variety of reasons inaccuracy or obsolescence of the 
source of the data, inaccuracies in listing, differences of definition, or 
differences in the units of measurement All these possible differences 
will affect the actual size of the subsamplcs from the sample blocks 
However, none of these change the probability of selection of any 
dwelling in any block because that is kept constant (at 1 /30) by the 
product of the two intervals 

This form of selection has general applicability Let us suppose 
that we want to select a sample of employees of a factory made up of 
work groups which vary in size from 15 to 500 employees Now we 
may want to satisfy two conditions (a) To give every employee in the 
factory equal probability (say 1/30) of being selected Thus, the ordi- 
nary sample mean will be a proper estimate of the population mean 
for the factory (i) To obtain a constant size sample (say 10 employees) 
from each selected section Thus, the analysis dealing with sections as 
units will be facilitated and sometimes made more efficient We have 
again an equation denoting the probabilities of selection in the two 
stages that looks like this 

A vl® = - 

300 ^ P, 30’ 


Pi IS the number of employees in the section or the estimated number 
m so far as exact numbers are not available In tbe equation, the 
number 1/30 satisfies requirement (o), the fraction 10/T, satisfies re- 
quirement (3) cither exactly or approximately, and T./300 for the 
selection of groups is necessary to make the equation hold 

The procedure will begin with the listing of all the sections The 
order of listing may be utilized to yield stratification in connection 
with the systematic selection which will follow Note that a section 
will be selected into the sample for each 300 employees Cuniulate the 
numbers of employees from section to section ending with the total 
number (12,000) in the population -^ke a random num^ horn 1 to 
300 and apply the interval of 300 There will I*' “ 

selections m^e But a section larger than 300 may W' 
more than once If a section falls in twice, make a double selection 
from u 
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The sample mean is again the simple Zx/n Incidental!), if in a 
sample of this kind 10 out of the 30 sections had a certain character- 
istic, It would be wrong to estimate from that datum that 1 /3 of all 
sections in the factory were similarly characterized Rather, one should 
estimate that 1 /3 of all employ ees worked in sections hich had that 
characteristic The estimated \anance of the sample mean is discussed 
n Section 18 


Selection with probabilities proporUonal to a measure of size is a 
^MdeIy used procedure With its use the size of the subsamplcs can be 
stabilized in cases where the cluster size vanes, in so far as useful (but 
not necessarily exact) measures of size are available The probability 
of selection through the t^^o stages is kept constant by the use of 
sampling rates and u is not affected by the nature of the measures 
ot size or by their inaccuracies 


This procedure is used 
are used as sho^vn in the two 
tion may be made from each 


m practice in t\vo ways first, intervals 
examples above, secondly, a single selec- 
stratum, as in Section 21 


21 A Multistage Area Sample 

bas,c’s'‘amnl,!!!‘°'T® for selectiug the 

aen7ed'^afr„ exam r T’r™ I' P^' 

ampt T^e 7 ^ scale mulmtage area 

and listed in each stratiim tt. «nits are placed into strata 

tn the stratum are ar;:ed^:::'*' 

proportional to its measure of SKe"^"*' “ P‘'°‘’^““>’ 

units there occurs a repetition of the ^ selected 

and listing of units and the a P^xtesses, the defining, stratifying, 
selection (7) ss'gnauon of the measures of size and 

units* of the Unitrf'tlte° d^' “if'™ 

shall discuss a basic design adequate 

*The sample of private dwellum n i d 
chiefly hotels and institutions. They could ? j residences of some people, 

separate stratum. The 40 milhon fieure suitable procedures in a 

this sample ^ the 1940 data used for drawing 



Soldction of the Sample 231 


for a sample of about 4000 d\s cllings, a sample in which every dwelling 
IS to be given a 1/10,000 chance of being selected But it is understood 
that the same materials can be, and are, used over and over to yield 
various samples at different rates Area sampling is used— the selection 
of area segments determines the selection of the duellings associated 
with them 

The field work is done by part-time interviewers who cover, 
usually by automobile, an area within a reasonable radius of their 
homes An interview “load** of about 50 sample dwellings per sample 
area (for one or two interviewers) was taken on the basis of considera- 
tions of the economy of clustering (Section 1 4) But 50 sample dwellings 
represent 500 000 dwellings in the population, enough to populate a 
state So that the interviewer will not have to cover an area as large 
as a state, the sample area of each interviewer was to be restricted 
The county was chosen as a basic unit providing an economic com- 
promise between the aim of reducing the cost per sample dwelling and 
also spreading the sample over the diverse groupings m the population 

Most sample counties contain a city (usually the largest urban 
center in the county and near its center), several smaller incorporated 
cities and villages, some unincorporated congested areas mostly around 
the cities, and, surrounding all these, open country where farmers as 
well as many nonfarmers live The central city is the most likely place 
to find a suitable interviewer Usually half or more of the county s 
population (also half or more of its sample d\\ ellings) are m or adjacent 
to the central city The interviewer travels to the others (the outlying 
cities, villages, and open country segments) on roads which tend to 
radiate from her city With an automobile she can go to one or more 
outlying sample points and return home the same day 

Thus, the county appears as a rather desirable sampling unit for 
the first stage of selection, as a “primary sampling unit ’ There are 
exceptions to the idealized picture drawn above Some counties are 
too large or otherwise difficult to cover as an entity These may be split 
into smaller units In other cases, it is possible to build up from two or 
more counties, or from parts of them, primary sampling units which 
will have the desirable features mentioned above From the point of 
view of increasing the spread of the sample over diverse areas, it is 
best to combine into one primary sampling unit contrasting kinds of 
areas But to reduce the cost per sample dwelling, the amount of travel 
Within the primary sampling unit is to be kept down 
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and several other characccnstics (7, 13) One particular stratum con- 
tains only three p s u ’s, each with one of the large cities of the South- 
west On the other hand, the county exhibited below is one of more 
than 100 p s u ’s that compose this stratum, largely from the rural 
parts of the Plains States 

The selection of the p s u was made from a list of all the p s u ’s 
in the stratum, on which were cumulated the measures of size of those 
p s u ’s (their 1940 populations) A random number from I to 1,831,704 
was taken, since this was the total 1940 population of the p s u ’s in 
the stratum The number chosen was one of the 19,277 numbers 
(between 253,613 and 272,889), any of which would have selected this 
particular p s u into the sample 

The aim was to assign to every dwelling in the United States— 
hence also to every dwelling within each stratum— a probability of 
1/10,000 of being selected From the stratum, a p s u was selected and 
Its probability of selection was 

19,277 1 

1,831,704 95 02 

What should be the rate of selection of dwellings within the 
county^ It should be 

1 1 1 
10,000 95 02 105 24 

because thereby the probability of selection of any dwelling in the 
stratum through the two processes of selection is made 

95 02 105 24 10,000 

The requirements of probability could be met simply by listing all 
of the more than 5000 dwellings ^ in the p s u and applying the interval 
of 105 24 to It However, that procedure wasjudged to be too expensive 
Alternately, one could divide the entire area of the p s u , or county 
in this case, into small areas, segments m the open country and blocks 
m the towns, and apply the interval of 105 24 to those areas That 

tWe obtain these crude estimates of dwellings, as quick aids in planning the 
field Work, using a factor of 3-1 /2 or 4 persons per dwelling for the total of J 9 277 
People in the p s u 
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would be a feasible procedure m this largely rural county and suitable 
for some surveys For our surveys it was judged that the town blocks 
with about 6 to 12 dwellings were larger than we desired Therefore, 
It appeared that some subsamphng was to be done within the p s u 
Maps, air photographs, and census data were procured to sec 
what the county contained It was decided that trasel to a town was 
to be made for not less than 6 sample dwellings This minimum for 
the sample yield in the towns (the secondary sampling units) is an 
economic consideration similar to the assigned yield of about 50 
t sampling units (Section 14) It means that a 

6/sn —'To county should contain no less than roughly 

the ,h7ee /r'" ““"‘y divided Into 

oUiee LI' •*'' populations for measures 

of size Each of the strata is to be sampled with a rate of 1 /105 24 In 


Composition or a Sample County 


Description oj stratum 

Measure aj 

Silt {.W40 pop) 

Cumulated 
measure ej 

Central city only urban place 


size 

3920 


All smaller incorporated places 

872 

872 


584 

1456 


447 

1903 


443 

2346 


357 

2703 


224 

2927 


171 

3098 

Open country (the remainder) 

3098 

12,259 


Total for County 

19 277 



dividing the eounty into strata and . 1 , . 

principles of listing must be observed '‘'•™P'™S units, th 

only one sampling unit (Section, e , “PP”'' ™ 

clearly and consistently before set T"* division must be madi 

Stratum I consists of the one my"!! 
of blocks and dwellings, as dcscnb~7 1"“^ sampled m the two stage 
in Section 19 The product of th< 
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probabilities of selection for the two stages was 1/17 54 X 1/6 = 
1/105 24 Altogether, the probability of selection of a dwelling in that 
city depended on three selections 

^ ^ X ^ ^ 

95 02 17 54 6 10,000 

In Stratum II it was desired to restrict the expected yield of about 8 
dwellings to one of the 7 villages which composed the stratum The 
random draw from 1 to 3098 selected the fourth villagewith a probability 
of 443/3098 = 1/6 993 In that »own the rate of selection had to be 
madel/105 24 — 1/6 993 = 1/1505 This was done again in the two 
stages for blocks and dwellings as 1/5 02 X 1/3 = 1/15 05 Thus a 
dwelling in that town was selected in four stages (county, town, block, 
dwelling), and the product of the probabilities is 1 /95 02 X 1 /6 993 X 
1/5 02 X 1/3 = 1/10,000 

All of Stratum III, composed of the open country surrounding 
the other places, was divided into area segments ^ There were 736 of 
these segments with an a\eragc of about 4 or 5 dwellings per segment 
The interval of 1/105 24 is applied directlv to those segments, and 
every dwelling m the selected segments is included in the sample 
Thus, about 7 segments, with about 30 dwellings m all, are expected 
from the open country The selection of a dwelling in the open country 
IS done m only two stages 1 /95 02 X 1 /105 24 = 1 /10,000 

The work performed m this county was similar in nature to work 
performed separately in each of 66 primary sampling units The pro- 
cedures were suited to the circumstances found in the pnmaiy sampling 
unit It should be noted also that the expected numbers of sample 
dwellings are not “quotas ** The selections are made with sampling 
rates, which are numbers controlled to yield sample dwellings with a 
fixed probability In so far as the actual populations are different lioni 
the measures of size, there will be variations in the yield of the sample 
from different areas 

22 PROCEDURES OF ESTlMATIOI\ 

3Ne ha\e examined in this chapter the role that some practical 
methods of selection can plav in improving the sample design The 

tThu division, as well as other useful materials, may be bought cheaply from 
the Census Bureau and is part of the Master Sample 
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sample result w.th which we dealt (except ,n Sections 11 and 17) was 
the simple mean of the sample Sx/„ But in all cases we understood 

esurna^' f 5 ™ “ 

estimate of the mean of the population which the sample represents 
Sample design was defined earlier as dealing jointly with pro- 
baseT:„°V The precis™ of the «l"a“e 

.hrour,he 1 c? ■"■P'^-'d-sometimes considerably- 

ompfex ftr information The topic is too 

he reade " ™P°«ant. and the attLtion of 

the reader IS called to It (l.pp 111-188. 22. pp 145-182) 

of 400 out OM 'r"'!''’ *" f random sample 

Suppose^Lf we’rveTh^'Xmm™ “ ■" 

employees that belong i„ the severa ' l" ^ P™P“«.Pns (N./N) of 
we did not use that iLoriT . r “ variable Although 

process, after the data have been coll ‘*'"'‘"8 the selection 

tion to improve the sample mean ““ *' mforma- 

that information not used in the first'^raceT‘'°" 
selection so as to obtain a P'ace for stratification during the 

those classes’ Well oerham stratified sample from each of 

' 'I'd not think of It at the ttme 


Number m class h 
Proportion in class h 
Numbers m the sample 
Numbers of answers 
Proportion of ‘ yes ’ answers 


union tells us that of all the causes Suppose that the 

belong to the union But union~^^ ^ factory 7200 

payroll cards used for the sclectint.^^*^*^^*^ indicated on the 

from the union, or from the respond ° ® sample Suppose, also, that 

in the sample belonged and "^hoof the 400 

definition of belonging as ascertained n. It is necessary that the 

c *bc sample cases be the same 



Total 

Union 

PiOnumon 

Nk 

12 000 

7200 

4800 

= N,/N 

1 00 

60 

40 

«* 

400 

260 

140 

Phrn, 

80 

26 

54 

Pk’ 

20 

100 

386 

many straa on the basts 

of other variables 

not ■" the selection 

not available beforehand 

process 
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as that used to classify the population into classes The pertinent data 
on the eights for the two groups are given below, together with the 
results on the number of “yes” answers to a question from a simple 
random sample of 400 employees 

The proportion of union members m the population is fVi = N,/N = 
60 With proportionate stratified selection, 60 X 400 = 240 of the 
sample of 400 would have been selected from these, actually 260 were 
This is not a large deviation, but the chances of getting a larger 
deviation are only about 1 in 20 * 

Now the ordinary mean p' = 80/400 is, in effect, one where the 
strata are weighted in proportion to the representation in the sample. 
That is, 

/,'=?“ X — + — X — = 65 X 100 + 35 X 386 = 200 
W ^ 260 + 400 ^ 140 

However, if we use the information we possess about the correct 
weights of the strata, we have 

p„' = 60 X 100 + 40 X 386 = 214 

We see that the correction is not great in this case, particularly in light 
of the standard error of 02 

It 13 more productive to look at the matter in probability terms, 
as wc did in the case of the proportionate sample in Section 9 The 
weighted mean p„' in this case has a variance about 2 percent less 
than the variance of the unweighted mean p' That is, by weighting 
the sample of 400 we make it as accurate as a sample of 408 wouliLbe 
Without weighting The gams in precision obtained by this kind of 
weighted estimate arc just about the same as they would be by propor- 
tionate stratified selection procedure This should not be surprising in 


■It must be noted that this last fact o m sharp eontrast ^ 

adjustments by which conformity to population proportions is 1 °"““ “n da a 
obtained in some haphazard fashion The latter may have 0^™““ ^ 
different from the popnlation, hence the •■correeuons " may b' ^ 4r^“' 
means that the “coveted” mean may be very -hoobo “'r ” “ 'b- 

than the uncorrccted mean However, because the da a a mean to 

abiUty sample, statistical theory will not bridge the gulf f™” „„ 

population mean The conjecture -”1' bf 

possible that the adjustment took .he “eorreered” ^^ 0 ^ 

population mean Furthermore, no probabihty can be attached to the oecurrenee 

of thu undesirable event 
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V.CW of ,hr fact that the method of est.mat.on in the two processes ts 

obtar/’ T”' "’■= ’"■■"ber of actual cases 

t.on«e sam 7?°” ” 'he propor- 

tionate sample of elements, and for the same reasons 

auxihar! , r'’™ " ■l>o«rat.on of a procedure for using 

sl .n nrec' m '' '* ^ P°“ 'Hostrat.on in that the 

Sp" ^abk m oth’' o«tn,at.on procedures 

may ^e made The '''' '^^8= S'""* 

of better estimates " ‘“amnces, the gams made through the use 

better estimates are spectaeulard.pp 111-188, 3, p 87,2i,p 155) 
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PART III 


Methods of 
'Data Collection 


In terms oE a gross division, there are only three methods 
oE obtaining data in social research one can ask people questions 
one can observe the behavior oE persons, groups or organizations, and 
their products or outcomes, or one can utilize existing recor s or 
data already gathered Eor purposes other than ones own research 
The last three chapters in this part describe techniques to be used 
m connection with each oE these broad classes oE methods The first 
chapter is a theoretical analysis oE the problems oE measurement 
In social psychology, emphasis is placed upon met ° ® ° 
collection in svhicli the researcher liimselE introduces the specific 
tions and the controls, and the chapters on -"''rviewing and be 
hasioral observation address tliemsehes to this problem The useful- 
ness oE existing records for many research purposes, howeier, is 
demonstrated in Chapter 7 



mcihodolt ‘ “"d "hat IS not 

areas Prll t'' '''^ “ Psychology and related 

the realm if •v7n t"“’T‘ ™”P'c. ordinarily considered m 

on iicS Irt? have been used to good advantage 

in tie Even techniques®, vhich belong 

attitudira“d tiiui P^>'^hology can be used in research an 

possible for th^ book to obviously neither desirable nor 

ment techniques ThrS “ r" 

all the methods whirb u ^ attempt to describe not 

research but rather those TOthidTirm?'''’’ 

eogaged in researeh m this area ca„„ra™r“‘ 



CHAPTER SIX 


Problems of Objective 
Observation 


Helen Peak' 


All scientists hate as an .deal the objectification of their 
meihocU and techniques That is to say, they aspire to ° ® * 

record, and interpret events m such a fashion that indepei^ent 
observers can verify their findings The 

ures” has been used rather narrowly by psychologists to apply to 

certain litniied hinds of ‘ tests" which are re atise y spec. 

procedures for administration, for scoring, and for 

of the scores on the basis of their rehabHity and 'f 

reference to established norms of performance t is . 

lamed that whether one speaks of standardized too s in 
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pose, they must be constructed to specifications In either case 
whether it be a matter of selecting an appropriate measurement 
device or of building one to order certain criteria are operating 
and certain questions are asked Xo make clear what these questions 
are and to summarize and criticize some of the ans^vers to them is 
our present task 

It will be convenient to organize the discussion around six basic 
problems 

1 What behavior is to be selected and recorded in order to obtain 
the information required? 

2 Under what conditions are observations to be made? How is the 
observational situation structured? 

3 What evidence is there that some process with functional unity 
is being observed? 

4 Has an attempt been made to summarize uhat i> observed in 
quantitative terms’ Can a score be assigned and what are the 
metrical characteristics of that score? 

5 ^Vhat IS the nature and meaning of the process which has ilius 
been observed or inferred? How is ii to be labeled’ What is its 
validity^ 

And finally, how stable are the observations? Can the *^ame 
results be obtained under what appear to be the same conditions’ 
Are the measures reliable^ 

The first two of these questions will be considered together in 
the Following section. The Observational Situation and the Selec 
tion of Significant Behavior Questions 3 and 4 will be disaisscd 
together m the section on Evidence of Functional Unity, because 
the demonstrations of functional unity are cIosel> related to pro 
*ems of scoring Questions 5 and 6 wd! be treated m separate 
sections 


the observation \l sn u \tion ^nd the 

SELECTION OF SICMFICXM BEHAVIOR 

Somcihing IS to be observed and meisiircd. and the .nscsiigaior 
start tvith h)potheses about tlic mj m "bicb tins process or 
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tions to be given, time limits to be used, and the questions to be 
asked Perhaps the failure to cope more adequatel) with other 
aspects of this problem stems from the implicit assumption that the 
processes being measured are independent of most variables and 
therefore relatively static and stable, an assumption which is clearly 
false for such processes as attitudes, needs, adjustment mechanisms 
interaction, and other group phenomena Here it is obviously just 
as necessary to have some notion about the existence of uncontrolled 
determinants as it is to know whether a patient has walked up a 
flight o£ stairs before taking a metabolism test 

Ideally, then, objective observation requires a clear indication 
of the behavior to be selected from the complex matrix of activity 
and definite specifications of the conditions for eliciting this be 
havior in such a way that inferences may be made about the 
variables involved But it should be noted that there is no simple 
methodological prescription for meeting these requirements ey 
ate the products of the theoretical sophistication knowledge, orig 
mahty, and hard work that are demanded in order to build theories 
about interrelationships between events and to test those theories 
empirically To identify the important variables to think of ques 
tions to ask, ways of structuring the situation, the appropriate 
behavior to observe-these steps depend to an important extent on 
the flash of insight and the hunch founded on knowledge and ex 
perience with the problems under considei ation The rules for 
the formulation of questions and statements, which are found m 
methodology texts, are simply convenient criteria against which to 
test ideas but have nothing to do with producing them And it v«ll 
be apparent that here, as at many other points, the theoretical model 
'vhich the investigator brings to the task ivill play a crucial ro e, or 
>t will be a major source of the ideas "wluch occur to him an o t e 
choices which he makes If, for example, he sets out to devise a 
measure of hostility with a knowledge of the psychoanalytic theor> 
of defense mechanisms, the questions asked and the behatior o 
served will be very different from that which would seem relevant 
•f manifest expressions of hostility were regarded as the onlj appro 
pnaie data 
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mean that (I) they cliange concomitantly; (2) they are dynamically 
interdependent; or (3) one process changes dependently with an- 
other (cause and effect). Most of the methods for discovering the 
presence of functional unity deal simply with concomitant change; 
t.e., they derive in large part from correlational techniques and 
consequently are mute with respect to the existence of dynamic 
interdependence or causal relations. These distinctions between 
kinds of functional unity will be discussed further in the section 
on “Dynamic Organization.” 

The first task is, then, to describe and evaluate certain of the 
correlational procedures for demonstrating what is here called func- 
tional unity. This is not to inquire about what is being measured 
but rather about the existence of some process or some aspect of an 
event with sufPcient integrity that it may be identified as organized. 


The Problem of Scoring 

As we examine the concept of functional unity, it will become 
c^ear that it is intimately related to the question of scoring. Some 
method of assembling the evidence from multiple items of behavior 
into a composite is needed in order to reduce the raw data to man- 
ageable form, and scoring is one form of summarizing. It is possible, 
of course, in the simplest case, that the data ivill be a correct or 
incorrect answer to a single task, an agreement or disagreement with 
a single statement about an isolated issue, or simply the fact that 
^ child either got into a fight or did not under some condition, 
typically, however, many behavior items will have been noted, 
and these must somehow be drawn together. This may be done 
intuitively and the result expressed verbally without benefit of 
numbers, but it is clear that where procedures for combining data 
into scores are definitely prescribed, errors of subjective interpreta- 
tion and bias are eliminated in some degree. Furthermore, there are 
many obvious advantages in numerical indices, always provided they 
do not distort the data. These problems will be discussed below. 

In any scoring procedure, several steps are involved, and the 
first of these requires that the items of behavior which are to be 
Combined into a score must be classifiable into tlie same category 
on some grounds. This may involve simply an a priori decision 
that all the items in a test do, in fact, fall in the same category— 
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use with these scores? How do we best express the relative scores 
of persons when comparing them with others? 

The answers to these questions open up the important problem 
of the relationship between the model implied in the use of mathe 
matical operations such as addition, multiplication, correlation, 
etc , and the data to which the operations are applied Coombs, in 
Chapter 11, has described the characteristics of various types of 
scales, and the reader is referred to that discussion in this connec 
tion The important point is this If we impose on a set of data 
assumptions, such as the common one that equal steps between 
numbers assigned to reactions represent equal intervals or mere 
ments of some psychological process, the results that emerge from 
the mathematical operations may be determined more by this as 
sumption than by the nature of the reality that is being measured 
If we assume unjustifiably that our scale positions are equal psycho 
logically and proceed to manipulate the numbers in accordance 
with these assumptions of continuity and equality, the resulting 
means and correlations will not reflect an accurate picture of the 
processes inferred to exist 

Specific Procedures for Determining F unctional Unity 

Certain operational procedures will be discussed m this section 
from the point of view of the evidence of functional unity i\hich 
they provide These procedures differ in a number of respects, in 
eluding the level of behavior complexity to i\hich they are adapted 
Those to be discussed first look for unity and organization among 
relatively simple behavior items, such as responses to single questions 
or statements Here we consider the analysis of internal consist 
ency among items by item analysis and by the demonstration of 
some rational order among behavior items by the scaling techniques 
At the next level the units or segments of behavior are total test 
scores correlations between tests are studied for evidences of 
organization At a more complex level factor analysis seeks simplify 
>ng relationships among intertest correlations Finally, certain spe 
cial problems of establishing dynamic unities will be discussed, 
ihcse are typically complex organizations 

For each of the methods of analjzmg functional unity, \\c shall 



252 Molhods of Data Collection 

raise the following questions ( 1 ) What, briefly, is the method? Hon 
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are used, weights ranging from one through five are assigned, so 
that extreme agreement with one item gets the same weight as 
extreme agreement with another item The score is the sum o t e 

weights , 

Many other methods o£ assigning weights have been worked out 
(25, Chap 18) (28. Chap 7. Supp Study B-4 and D) For example, 
weights may he determined on the basis of the correlation of each 
Item with some criterion to which the test is designed to predict 
This often means that the weights must be changed when predicting 
to different criteria Or weights may be selected so that the disper 
Sion of the individual scores is a maximum But the difficulty with 
these and many other methods is that they do not commonly attempt 
to offer any theoretical reason concerning the manner in which the 
processes reflected in the items actually work together to pro iice 
the predicted consequences 

These correlation techniques have serious inadequacies as tests 
of functional unity among the items of a lest, whether t ey are 
applied to the analysis of relationships between items or etween 
Items and total score In the first place, some of the met o s run 
into problems regarding statistical assumptions Indices, like tetra 
choric and biserial r, assume linearity of the relation between vaii 
ables, normal distribution, and continuous variables It is impossible 
m some cases to determine whether the data conform to the statisti 
cal assumptions-for example, the assumption of continuity m tti 
chotomously answered item variables In other cases t e require 
ments may be clearly inappropriate to the data, as when tetrac one 
r IS calculated from tables having a single cell with a zero entry 

In the second place, the interpretation of these indices as in i 
cators of the degree of functional unity among the items of a gisen 
instrument is often ambiguous There is for example, no satisfactory 
way to disentangle the combined effects on correlation of the pres 
ence of items at various levels of difficulty and the degree o letero 
geneity of processes in\olvecl in responding to the items o t le tes 
Some indices, such as phi and point biserial have arti cia im 
set on their size by the difficulty of the items correlated The maxi 
mum value of phi, for example, is unity only when the same proper 
non of people pass both items involved in the correlation But it is 

ISce Locv,„„cr (32 PP 17(1) (or =■ "t ll.cvc ind.w, of ilcm 

'alulitv 
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factor) and ambiguous evidence regarding the existence of func 
tional unity among the items of a test Each item may be correlated 
to some extent with every other item and with tlie total score but 
this still does not mean that all the items involve the same process, 
even to some limited extent The existence of correlation between 
pairs of Items is a necessary but not a sufTicient condition of the 
existence of unity between all the items that make up a test Finally, 
nothing is revealed about the nature and complexity of the processes 
measured by the instrument made up of items thus selected Theie 
IS only evidence that some of the items are related, but because 
of statistical ambiguities the extent of this relationship is uncertain 
At best, the methods are appropriate to the initial stages of analysis 
of this problem of functional unity or to the selection of items of 
a test to be used chiefly for prediction to a specific external criterion 
ANALYSIS or INTERNAL CONSISTENCV BY RATIONAL ORDERING OE 

ITEMS Here, as in the preceding section, we shall be concerned 
with Item analysis But the scaling methods go be)ond correlation of 
Items and make the attempt to order the items with respect to one 
another— i e , to assign scale positions to them Certain of the scaling 
methods proposed by Thurstone, Guttman, and Loevinger will be 
examined in order to determine the contributions of these tech 
niques to the solution of the problem of functional unity ® 

In the construction of Thurstone s equal appearing interval 
scales the selection of items and the process of scaling proceed 
simultaneously Many statements about a given subject are pre 
pared, and judges are instructed to sort the statements into eleven 
piles considered equal distances apart and ranging from most favor 
able statements at one end to least favorable at the other en wit 
a neutral position in the middle On the basis o£ these sorts, it is 
possible to determine the mean or median position assigned to 
each statement by the various judges and the interquartile range 
(0) of the assigned positions Statements with low Q_ values and with 
median scale positions approximately equal distances apart are 


5 These have been chosen as types of scaling 
the important poinu to be considered The most rmportant 

been oirntted rs Coomb, ordered and partrally ordered mettre toed on mull^le 
compansons rvhrch rs d.scussed rn Chapter 11 “d 

Patrick have suggested an ingen.on, combination of the Thurstone Likert 
Guitman procedures (16) 



256 Methods of Doto Colleclion 

deviationrwith™"*"*^'^? appropriate to use means and standard 

icallv eaual r>ifin ’ implies of course, that numer 

bf regared as ind P°'"‘ ‘he scale may 

I.m”emelemd hi'"’"''' T”""" - attitude' 

remains doubtful tor a numh "'’ ' ‘‘““'"P““" 

consider the eleven piles rbel“ 

not mean that the Jr^ equal distances apart, this does 

on the unsophisticated .itH ^ equal distances This places 

only the relative amoum responsibility for interpreting not 

-th certain «aLrtsr/rret!r“' 

In other words he mn«f t. *ncreiiients of unfavorableness 
involved what the experts hX** Jtnowledge of the problems 

I' « essential a^Te „ ^ ""P““>>le 

mg theoretical lustificanoif? should be a convinc 

relation of an inferred nrorA.'"^ making any assumption about the 

theory is wholly lacking Moreover^'ihe''™"'’'’’"' ‘“P°'“^- “"th 
the scores obtained bv iHp inevitably distortion in 

the scale because of the restnrtT^^ extreme ends of 

A score of eleven (maximuTn\ ””8^ of items at these points 
With the one most extreme itern Tf °^tained only by agreeing 
median will be pulled m tow rt ®‘^*ier items are checked, the 
1 ely that there is distortion m^th ^ of the scale It is also 

thernselves due to what has k W judges 

end effecl This is ihe lendenr , ■" Psychophysics Is .he 
end categories (23) and distribu.m '‘’V“'‘8™c"‘s to pile up in the 
soring method suggest that th by the Thurstone 

about equal mtervaU should '■Wens Assumptions 

he distinguishing chara^-»« regarded with skepticism 

judges and not respondents deip*^****^ ts then that 

mg o Items and consequentlv meaning and the order 

variable This involves the assum^f assigned to the inferred 
?=--/^--ments3 IS some common 

a stracted The dimension mav h« dimension can be 

y ^ favorableness toward an object 



Problems of Objective Observation 257 


as in the attitude tests, or excellence of handwriting, or amount 
of threat implied in certain actions, or cohesiveness of a group, or 
anything else The evidence that the task is a feasible one and that 
some such dimension does exist rests largely on the fact that sub 
stantial agreement among judges can be reached regarding the rela 
tive scale position of items Moreover, a number of studies have 
seemed to show that statements scaled by the Thurstone method 
maintain their scale positions even wlien persons with widely differ 
mg attitudes serve as judges (19, 27, 36) But as Carter (7) has indi 
cated, this question cannot be answered once and for all, and it is 
necessary to take the problem of sampling into account when choos 
mg judges to be used in the standardization process 

Furthermore, scale positions have been shown to be affected by 
other conditions Farnsworth, for example found marked shifts in 
position of Items on the Peterson Attitude Toward War Scale from 
1930 to 1940 (17) More recently, Hovland and Shenf (30. 37) have 
pointed out the inconsistency between the two sets of findings (I) 
that assigned scale positions and judges attitudes are independent, 
and (2) that perception and judgment are affected by motivational 
and attitude factors They have reopened the entire question by 
an investigation of the influence of judges attitudes on the positions 
and distributions of items sorted by the Thurstone method The 
judges used were Negroes, pro Negro whiles and anti Negro white 
groups There is evidence of marked piling up of statements m 
unfavorable categories by Negroes and pro Negro whites and in 
the pro Negro categories by anti Negro whites It appears houever, 
that even in this experiment there is no grave distortion of relative 
Item positions The implied equality of units is meaningless and 
perceived distances between items are greatly altered by the judges 
own attitudes, but roughly the same items still appear to be more 
favorable or less favorable than others to all the groups The rho 
between item rank assigned by Negroes and anti Negro white sub- 
jects IS 0 937 

The question inevitably arises as to whether individuals work 
Within the same frame of reference when they take the test and 
whether respondent reactions give evidence of the same ordering 
of Items as do the reactions of judges Thurstone himself was inter 
ested in this problem and in the late 'twenties devised what he called 
the index of similarity or rele\ance, based on logic which is 
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ciently relevant to belong in the scale must be a matter of judgment 
Moreover, there must be as many graphs as there are items, and no 
method xs provided for representing the homogeneity of the test 
as a whole 

It appears that Thurstone has not made extensive use of this 
technique in the construction of his scales Edwards reports a com 
munication from Thurstone m reply to an inquiry on this point 
which states that he has come to feel that items should be selected 
on the basis of factor analysis rather than by the criterion of simi 
lanty (15) 

In any case, it is clear from a number of studies that the Thur 
stone scales, when tested by the index of similarity, often contain 
statements which do not maintain the scale position assigned by 
judges Edwards (15) has shown, for example, that the neutnl items 
in a number of Thurstone scales are likely to be accepted by per 
sons who agree with statements widely scattered on the scale Dudy 
cha produces similar evidence in a study iv'hicli reports the analysis 
of the scatter of endorsed statements on the Peterson Attitude 
Toward War Scale (14) He found that individual students t\ith 
liighest and lowest scores often showed endorsements of statements 
ranging from strongly favorable to strongly opposed, and he ques 
Hons the value of medians derived from such widely scattered scale 
values 

The fact is, then, that the functional unit abstracted out b) the 
Thurstone method of equal appearing intervaJs is not usually the 
same as that which is found by scaling techniques based on respond 
ent reactions, and the question will be asked whether the organiza 
Hon found by one or the other of these methods is the more truly 
fundamental The answer must be given in terms of the efficiency 
of the constructs which the measures yield for specific purposes 
If. for some purposes, constructs based on scales which are ordered 
in terms of respondent answers produce more verifiable predictions 
lo behavior or to other constructs, such scales can be regarded as 
superior for these purposes The possibility exists that judged scale 
position of statements may prove to be useful for quite different 
purposes, many of which have not been adequately explored Such 
consistently ordered series of items might, for example, be regarded 
as measures of group norms regarding the implications of certain 
statements rather than as a basis for accurately inferring an attitude 
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reproduced from knowledge of the total test score. In the cumu 
lative type of test, this will be the case when it can be shown that 
any person who agrees with a given item will agree with all less 
extreme items and will disagree with all more extreme items. The 
same score cannot be made by persons who accept different state- 
ments. For the differential type of scale, respondents will accept 
a group of items which are adjacent in scale position, disagreeing 
with those statements on either side of this position.'^ 

The details of Guttman’s method are described in Volume IV 
of the American Soldier Series (40). Stated briefly, the procedure 
involves mapping respondent answers to each item onto a scalogram 
in which items are placed horizontally across the top of the table, 
ranging from those most favorable to those least favorable. Re- 
spondents are ranked on the vertical margin of the table, with 
individuals giving the largest number of favorable responses at the 
top. After these preliminary steps, items and respondents are shifted 
in position until the maximum order possible with the given set 
of responses is discovered. The ideal sought is a pattern in which 
any person A with a higher total score than any other person B will 
have agreed with items as high as or higher than B. If there is 
no scale, items agreed with will scatter all over the scalogram 
without relation to the rank order of the total scores and no rear- 
rangement of items or persons can reveal this kind of order. 

The extent to which a given series of items departs from the 
ideal of unidimensionality is expressed by the coefficient of repro- 
ducibility. This index is simply the proportion of item responses 
that can be correctly predicted from knowledge of the scale scores 
of persons taking the test. When the scale consists of 5 items which 
have been given to 100 people, the total number of responses will 
he 500. If 50 errors are made, the coefficient of reproducibility will 
he 0.90, which is set as the minimum value permissible for a ' scale 
(*^0, p. 77). . f 

Although this is the principal criterion for the existence of 
onidimensionality, Guttman suggests that at least four other factors 
should be taken into account in making a judgment on this mailer. 
“Two of these should be mentioned briefly at this time. The items 
of the test should cover as wide a range of marginal distributions 
possible, and items showing a 50-50 distribution (i.e., 50 percent 

•See fooinote 6. p. 258. on aimulathe- ant! difTcrential-tjpe tests 
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reached on the intensity dimension is regarded as the zero point 
^\hich divides the favorable from the unfavorable side of the scale. 

Two comments should be made about this function. First, it 
IS not altogether clear that it can establish a division between 
favorable and unfavorable attitude responses in all cases, for not 
all data show this U-shaped function and those which do typically 
have a great deal of scatter about each of the points— i e , man) 
people falling at the middle or neutral point of the scale v\ill feel 
strongly. Second, even when there is a U-shaped curve, the zero 
point cannot be regarded as an absolute zero where there is a com- 
plete lack of attitude or of affect toward the object of the scale 
The determination of such a zero point, like the discovery of equal 
intervals, must involve some theory about the way in ivhich the 
variable being measured is reflected m the observed behavior 
There is no such adequate theoretical rationale as )et 

It is assumed by Guttman that “the invariant cutting point 
between favorable and unfavorable responses” can be determined 
by this method and that this point will be independent of the bias 
resulting from the specific questions asked. In other words, it is 
asserted that the percentage of persons favorable toward tlie issue 
or object will remain the same from one set of questions to another 
when the point of shift from favorable to unfavorable is determined 
by selecting the score corresponding to the lov\est point on the inten 
stty scale. Guttman provides some evidence that this is true, but it 
remains to be seen whether this location of zero has an) vahdit) 
the sense of being related to a point of shift from favorable to 
unfavorable action. 


her of 


The Guttman method of scaling can be evaluated from a 


points of view. First, questions have been raised regarding 


ue ambiguity of certain steps in the procedure ^Ve have indicated 
that Guttman recommends the application of se\eral criteria in 
addition to the coefficient of reproducibility in judging the scala 
bihi) of items. It has been pointed out that tlie application of these 
t^tteria is essentiall) subjective and that there are no clearl) pre 
^CTibed rules for arriving at a final judgment. For example, u must 
e difTjcuIt in practice to determine mercl) by inspection vi.hcther 
errors are random or s)steniatic, )et this judgment is one step in 
eciding whether the data reveal the presence of a scale, a quasi 
*eale, or a nonscale t)pe of test. Likewise, the problem of the 
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indicates that scale analysis affords a rigorous test of the existence 
of a single meaning for an attitude area in the eyes of the respond 
ents This is an acceptable interpretation but Guttman goes on to 
make the further assumption that when a scale is shown to exist 
among items, the universe of which the items are a sample will also 
scale, as will any other set of items from that universe On logical 
grounds this statement must also be accepted, but this is not the 
same as saying that we can in practice identify that universe from 
which scalable items come Guttman says that a uni\erse must be 
defined in this fashion 

An attribute or item belongs to the unnerse by virtue of its 
content The investigator indicates tbe content of interest by the 
title he chooses for the universe and ill attributes with that con 
tent belong m the universe The evaluation of the content 
thus far remains a matter that may be decided bv consensus of 
judges or by some other means (40 p 84) 

He points out, further, that samples from many universes so defined 
do not scale or are made up by several subscales Items which scale 
under one set of conditions and with one population do not neces 
sanly prove scalable m other circumstances It appears, therefore, 
that the most that can be learned from the demonstration of um 
dimensionality is that a given set of items scales under the specific 
conditions and with the specified sample Having determined this, 
We have no way of inferring directly what the universe is of which 
these Items are a sample Only empirical test of some theory about 
the reasons and conditions for the appearance of order among the 
Items will make it possible to identify other situations (items, frames 
of reference, populations) to which one may generalize the findings 
The question may be raised whether the method is adequate 
for discovering all kinds of fundamental psychological variables It 
may be appropriate mainly to the task of finding umdimensionality 
where answers to certain questions imply definite answers to other 
questions which are obviously related For example, if a person 
ts asked whether he wants to stay m the Army for a career after 
die war and answers, "Yes,” it is reasonable to expect tliat he would 
give an affirmative answer to the question ‘ Arc there any con 
tlitions you can think of under which you might consider staying 
m the Army after the war’ In such questions the dimension 
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test (//,) IS a ^\clgIuc(l a\crage of the f/,/s for each pair of items 
adjusted so that the cocfTicient equal 0 on tlie perfectly hetero 
geneous test and equal 1 for the perfectly homogeneous test 

The logic of Hu assumes that item ans^vers should rank two 
people in the same way that total score ranks them— i c , the person 
who passes an item should have a higher score than the person who 
fails It Hu IS equal to the percentage of such correct discriminations 
minus the percentage of wrong discriminations The relation be 
tween H^ and Hu is not known 

In selecting the items for -a test, it is proposed that all the 
values be calculated and that the items with low Hu 'allies be 
eliminated This follows the usual procedure of item analysis Loev 
inger then points out that if the heterogeneity (H,,) is evenly dis 
tributed over all the items, *we still cannot decide whether we 
have what Guttman calls a quasi scalable universe or two sub 
uni\erses To make such a decision we need a table showing the 
coefficients of H^ Apparently for a quasi scale these coefficients 
'vill be moderate m magnitude and fairly uniform In case [among] 
the 'allies of H^ [there] are some aery high and some ver) low 
despite uniform values of Hu, we expect there is a way of dividing 
the Items into two or more tests each of which is more homogeneous 
than the original test' (33. p 520) Loev inger goes on to indicate 
that It IS doubtful that items will of'en be easily separated into 
two or more homogeneous tests m this way, with s all close to 
unity or zero In other words although the calculation of these 
coefficients is straightfoiward and more objective than some of the 
procedures which Guttman recommends for interpreting the coeffi 
went of reproducibility, Loev inger is also faced with the difficult) 
of determining what shall be considered acceptable evidence of the 
existence of homogeneity or of a single dimension She recognizes 
this fact and suggests that it must be determined in the light of the 
purposes at hand A test which is sufficiently homogeneous for some 
purposes will not be good enough for others 

Loevmger has attempted to rule out the effects of difficulty 
on her coefficient of homogeneity by the device o£ dividing the value 
Pwj — p, by 1 _p, This amounts to expressing p,,, — p. as a pro 
portion of the maximum value it can attain at the given level of 
difficulty of the item As a result, H„ can vary from one to zero at 
any level of difficulty This constitutes some gain over the am 
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Whether such organization has any relation to the functional unity 
implied by scaling of respondent answers is a problem for empirical 
lest The available evidence does not suggest close correspondence 
between scales constructed by these different methods 

Scaling methods provide a more systematic and rational method 
of studying the organization among items than do traditional item 
analyses Another advantage is found in the fact that when items 
are shown to scale, a ration u method of weighting the items is 
provided As long as the weights assigned are equal or increase 
or decrease consistently with the scale position of the item, persons 
obtaining a given score will do so by answering the same items 
and a change in weights will not change the relative positions of 
scores 

The following important limitations of scaling methods must 

be kept m mind , 

(1) The confusion between the effects of difficulty levels and 
of heterogeneity of process on indices of unidimensionality or 
homogeneity has not been wholly eliminated The ambiguity whic 
this introduces into the interpretation of the coefficient of repro 
ductbility has been mentioned Loevingers attempt to correct for 
this effect yields an index varying from one to zero but the proper 
ties of the index between these values is not clear 

(2) It must be noted that in these, as m all other methods of 
investigating interrelations between items or tests, the effects ol 
variable error cannot be separated from the effects of heterogeneity 
of processes actually involved in the reaction This will be discussed 
in the section on reliability 

(3) The functional unity reffected in a set of scaled items is 
very evidently dependent on such conditions as the sample used and 
the structure of the testing situation A scale which exists with one 
sample and under certain conditions may disappear with a new 
sample and altered conditions, a fact which recalls the statement that 
the standard meaning of any instrument must be defined with 
respect to a set of clearly specified determinants 

(4) Scaling alone tells nothing about the character or comp exity 
of the processes uhich are responsible for the observed organization 
The observed functional unity may be due to a complex o actors 
'arming together under the specified conditions of the test or it mi> 
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The “split haU ’ and “alternate forms ’ methods of determining 
'reliability” become specnl instances of measuring functional unity 
by mtertest correlations A deliberate attempt is made, of course, 
to devise “comparable” stimuli or items, which presumably tap the 
'same” process, but the e\idence that this has been accomplished 
rests on the correlations obtained The same correlations become 
evidence of reliability There is no satisfactory way of separating the 
effects of the differences in test items m two test forms from uncon 
trolled conditions which introduce random variation into the 
responses (unreliability) The isolation of the latter is the purpose 
of reliability tests, which are discussed in a later section of this 
chapter 

A number of familiar problems again emerge as limitations or 
as sources of ambiguity in interpreting these mtertest correlations 
as evidence for the presence or absence of functional unity Here 
^gain the effects of difficulty level (or frequency of agreement level) 
of the test items may affect the correlations and obscure, or enhance 
inisleadingly, the true relationship stemming from functional simi 
laruies in the psychological process underlying test responses For 
example if one test is made up of items with which a large proper 
tion of the sample population agrees and the other test has less gen 
erally accepted items, the correlation between the two tests will 
be reduced by this fact The coefficient cannot equal unity even 
though the same processes are common to the i^vo tests (6) But under 
®^her conditions misleadingly high correlations may result from a 
confusion of the effects of heterogeneous process and different dif 
hculty Jevefj- J/ a ssrnp!e of chrWfviT six to fw’efcv? yesrs o£ sge were 
given a reasoning test and a test of motor skills each made up of 
Items covering a range of difficulty the positive correlation that 
'vould result would not imply any integration or interrelation of 

processes of reasoning and motor skills but would be a function 
^ most exclusively of the fact that tasks of greater difficulty fall 
'vithin the competence of older children whether the tasks involve 
I'casoning or motor response This calls attention to the importance 
of knowledge about and intelligent selection of samples when inter 
Preting the meaning of mtertest correlations as indices of functional 
It also suggests the desirability of equating instruments for 
I ficulty before testing their mterconelations 

The sample populations to which the tests are gi\en may \ary 
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m two ways-m the qualitattve characteristics o£ persons sampled 
o" these characteristics-and the 
smX Ti '>y t^tther difference The California 

fenm™ Correlations between ethno 

San Ouent ^“nomic conservatism ranged from 0 14 for 

ins iar " "'“ttng t^kss women 

poputTonrha mv r" -e higher for the 

dence that the swe® standard deviations, but there is also evi 
sampled The authoL'^'" “ ^‘'^ttted to the hinds of groups 

between cthnoccntnsin an? n’ iT '°w correlation 

be due to the inadequate L™' orref"'‘“‘^“ 
as a basis for evalua^ng politica/and “ 

This means then that fv.n. e ^ economic events (2, p 836) 
provide conclusions genertlrabu " t' 

there is empirical evidence or a ,1'’'“"'^ samples used unless 
knowing what the condittons of grounded basis for 

Hore, as elsewhere the de» ^ oralization to other samples are 
a function of the specific st^o , '■''^"°"sbip indicated will be 
■"g the relationship This selec, ‘ selected tor express 

the characteristics of the dat j ■" 'bo I'ght of 

f the data are esslntiallv "> be made of Ihem 

between the tests may be eL?ei^L““'''.'" ‘b^ relationship 

o individuals correctly classified '‘‘tup y in terms of the proportion 
the ^Sis of knowledge of their da “"t"" oategories of test B on 

‘tons will express the relLed^ell 'q- 

e statistical procedures are anpro quantitative and if 

^ determine whether a distribm.T?'' " “ P^^-ble. "s a rule, 
which*""°"^ normal distribution l” conforms to the 

should?'^ '“plicit in many of the “"d homoscedasticity 

Itr h,^ ‘bat even . hood. "^ehniques But 

f;;‘‘ bntions of scores expressed ' '‘““"’Pfons ire met for 
«.tute proof that the variables reneemd'''"^'”^ ‘b"' ^oes nol con 

If , This IS to say that m 1" ''ate the same 

Idd ? "7 , ™"*“tememr,r r'™ 

Tthe m fi" "“b 'artables to, df ™P'‘’>' “ "lathemalical 
in i cor. 'b^ tenres mfrtm°' ‘b'= characenslics 

a correhnon .he coeliicienu TcZ T"' '"'“b'es mvohed 

•“Of correlation based on these scores 
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may not accurately represent the relationship between the variables 
But as will be shown later, the ultimate test of the validity of any 
construct and of the measures which enter into its definition is found 
in the utility of the construct in the process of reducing the matrix 
of events to some meaningful order If constructs derived from cor 
relations prove to have value as parts of a dynamic system, this 
suggests that the mathematical model w appropriate in some degree 
It has been indicated that a major problem in the construction 
of any instrument must be faced when combining responses to a 
number of items to produce a total score The problem must be 
recalled briefly at this point because the particular solution selected 
m any given case will affect mtertest correlations and thus the evi 
dence of communalily in the tests The earlier comments on pro 
posed solutions to this problem are relevant m the present context 
Only one additional point need be elaborated here It was suggested 
that most methods of assigning weights lack a theoretical rationale 
for determining the manner m which items of observed behavior 
should be combined to produce a score The point is this the satis 
factory determination of weights to be given items in a test and the 
manner in which they are combined to produce a score should rest 
on knots edge of how the processes tapped by the items combine 
dynamically to produce effects Coombs (13) has summarized con 
cisely the way m which distinguishable processes may enter into 
combinations (1) Processes may be additive, so that a greater quan 
tity of any one will produce increased effects (2) They may be con 
junctive, that is, each must be present m some minimum amount 
before any effect is produced and no amount of one will compensate 
for the absence of the other For example, both knowledge and 
motivation are necessary in solving a problem (3) ^ 

d'sjunctive One does not occur i£ the other occurs This is best 
Illustrated by the substitute mcchanisms-oiz , hostility may be 
expressed by reaction formation, by direct aggression, or by dis 
placed aggression This means that some items may properly be 
added, but that others tihich are not additive must be combined in 
tfch a nay that they represent the result nhich actually emerges 
Until analyses of this hind are possible, the arbitrary scores obtained 
b' combining test items have limited uses and the meaning of the 
^correlations of such tests is ambiguous 

When all these problems have been taken into account, an 
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obtained correlation between two tests simply demonstrates that some 
unctional Iinit^y exists, within the limits set by the size of the coelH 
aent and by the ambiguities of its interpretation From this infer 
sThether r'" “ " “ “y ''hat the common aspect is and 

for tlLin^'TmeirgPlre 

IS for tpcr »r j ^ homogeneous process, whatever it 

C O inle s r T' ‘O *e common 

■actor unless the correlation approaches unity 

as evS’encToTf'r;.”'^;'''''”" has often been used 

not, however illuminTt^ existence of correlation does 

simplicity or'complexity oLrwh^ underlying process, its 

(with allowance for unreliab h.^ ” ^PP^^'h unity 

terms of the common process ”n1c persons in 

'hehghtofthehm'utmnssrbyZ^fo'ir'''' ""T ''' 

are mliucnced by the diffich i “f facts (I) Correlations 

relations are a function “ ^“'related (2) Cor 

expressing coiariations must 

®rc to reflect the true r^Kr PP*‘^P*‘iate to the test data if they 
"teasured (4) The weights aZtn 

relations and should be deiA g *0 rest items often affect cor 
■n which the psychological nr™ ’'"‘"'ledge of the manner 
to determine a composite resu^T^”*^* tapped by the items combine 

FACTOR ANALYSIS If 

attempt should be made or later the 

e ge about intercorrelations h ^ accumulating jnass of knowl 
tnanageablc number of dimen, " '"taller and more 

■"'"hes a tfemonstratiL bTmmh’ P™““ analysis 

were cerlain underlying p,X™''''7‘t“l procedures that if there 
of rest ■"tercaiTeh..„®P;;“”« or factors x, y, and a, a given .able 
and”'i° uuruber of factors It furnishes a snte 

and die aieiagc loading ""ount for the matrix 

, The assumptions in facmT “'h test 

hose .hat enter in.o the intopref r" “"“rn us here are 

tudiesas "teans of discos cring ft,' , " ‘he resulls of factorial 
e data of factor analysis arc®iabl«‘“r’‘“’' “"‘““y Processes Since 
assumptions underlying ihose stau? ° o°rreIal.on coefficients, the 
tf 'his IS a hurdle which <s Wms” '"'“’'etl at the outset, 

^ ’ 'snored There are, in add. 
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lion, the following assumptions basic to methods such as those of 
Spearman, Thomson, and Thursionc. (1) \Vhen the factors con- 
tributing to a lest score are iincorrelated, the correlation between 
the tests is equal to the sum of the products of the weights of those 
factors common to the tests. (2) The component processes combine 
by simple addition to produce the response made to the test. (3) 
Every person taking the test is assumed to possess every factor. (4) 
Finally, the weight of any factor on a test is tlic same for every 
person. 


^Vhether the factors resufiing from an analysis will be correlated 
depends on an arbitrary’ decision by the investigator to use or not 
to use orthogonal dimensions. If he does so, the statement made in 
the first assumption will follow from the nature of the mathematical 
relationships. The fact tliat it must be decided whether to regard 
ihe factors as correlated illustrates an important aspect of any of 
ihe attempts to discover functional unity. Although the data them- 
selves set certain limits on what can be discovered, the hypotheses 
yhich the observer brings to the analysis will also determine what 
IS found. Spearman and his group have argued that the factors 
should be uncorrelaied, whereas Thurstone, among others, has 
insisted that factors which are the result of the complicated and 


interrelated processes of growth and experience should logically 
show some correlation. Since either type of factor may be found with 
die appropriate methods, the decision to seek one or the other must 
be made in terms of hypotheses which are independent of the process 
of factor analysis. 


Although the second assumption— that component processes 
combine additively— apparently yields the same results as would a 
^iiltiplicative assumption (38), a question may be raised regarding 
oe adequacy of factorial methods for discovering all the kinds of 
oi’ganization in which we are interested. It may be plausible to 
assume that abilities suramate, but it is not clear that this is a 
justifiable basis for predicting (he combined effects of all variables. 
IS equally reasonable to assume relationships of conjunctivity 
disjunctivity. In any case, a plausible theory is needed before 
simple additive assumptions can be justified. 

It is difficult to obtain clear-out evidence regarding the justifia- 
dity of the third and fourth assumptions— that every person taking 
lest possesses every factor, and that the test has the same loadings 
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on etch facloi for each person When we are dealing with inter 
individual variation it appears to be impossible to do more than 
estimate an individual’s score on a given factor, and these estimates 
(44) In discussing this problem, 
o e reports the observation that different people use quite diverse 
methods of attack on the same problem This suggests that attnbut 
ing the same factor weights (averages of individual weights) to all 
theTnd ^ certainly produces distortion of the true situation in 

lest adl“ . a a s performance on the same 

"ciahts of ni r™ eecasions may reveal the comparatnc 

mayte clrt : -O-duals, tn so far as^actors 

(9 PP 27/7 ) ^ ^ “'’et' the same in these different individuaH 

lying facwtanalynsl'Tnumbt™^^^ ^ 
faced There ts, first of all the f ^ familiar questions must be 
the elTects of difficulty level P™’’’'"’ °f distinguishing 

Since factor analysts star, u P^eess similarity or difference 
there ts the ZZ' ZlT ? 

has been affected tn the mann°''d’” ' coefficients 

of Item difficulty in the corr'i ‘^“"'hed above by the distribution 
that factorial methods ma ,ri^ Furthermore, it is clear 

to difficulty level for no factors which are due largely 

o-mple,thatwhen' a ma.r“'‘‘7'' has shoun, 10? 

of a homogeneous test fbv i ^ *”^®'’oorrelations between the items 
are as many factors as thpr#. criterion) is factored, there 

t would be expected that difficulty m the test (18) 

evels of difficulty, common ^ matrix represent various 

tests at the same level of difr would be discovered in those 

separating difficulty factors fro ^ This may constitute a means of 
leading conclusions 'Others, but it may also produce mis 

Factorial methods, hlg ii 

by the samples used The fa«n shown to be affected 

in one sample will often be d.iT factor loadings discovered 

erent characteristics This ® ”^w population with dif 

produced by anything that mr"’'!. "'“‘'P^^ing. since "factors are 
tarmbles" («. P 25) Th.s mea„r,n into a set of 

istics of a sample providinir th<» /il.* r”' relevant character 

^ or the original factoring must 
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be known i£ we are to be able to specify t!ie conditions iindei which 
the obtained factors exist 

Another problem is found in the hct that .my inatiix may be 
factored into an infinite numbei ol different components depending 
on the method used In other words, no one solution is unique 
The choice of any set of factors is, therefore, quite arbitrary and 
will be made in terms of the theories and hypotheses adopted by the 
investigator (8, pp 281^) The factorial methods like others yield 
evidence of the existence of functional unity Something is common 
to reactions in different situations The lines of cleavage are found 
in different places by different methods of analysis, and the resultant 
slices of psychological reality are not always the same The decision 
about which products are best, what they mean and i\hether the 
processes are simple or complex can be determined only m the light 
of some integrated theory of personality and behavior and not by 
statistical analyses alone 


In summary, the principal puipose of factor analysis is to reduce 
•1 matrix of correlations to the smallest possible number of dimen 
sions in the interest of parsimonious description of the interrela 
tionships between the variables The factors discovered may lead 
to fruitful hypotheses to be tested bv experimental methods For 
example, it might be demonstrated that a factor in certain tests 
emerges under ego involving instructions and disappears when the 
■nstructions are changed Such studies would greatly increase the 
significance and scientific utility of the patterns of organization 
discovered by this method 

There are a number of sources of ambiguity in interpreting 
the results of factorial studies (I) The appropriateness of certain 
■assumptions underlying factorial methods may be questioned (2) 


‘0 Limitations of space as ivell as ihe practical difficulties J" 
m as present form ha^e led us lo omu a discussion of 
of Liza^sfelds prom.sing technique of latent structure 40) Its 

"lost important difference from the traditional methods of fac or """ 
the fact that it is adapted for use with nonparametne data 
'lie unwarranted .ssumptions about units of measurement ^ 

^='ther methods I, is apphcable to data collected b> the -^^‘hod of sing e 
'"■null Like other methods it demonstnies onl> th«e for 

'he existence of certain underlj.ng tra.ds It do^ not ' " 

"t^suremeni or determine what they are Coombs and h's st udenu arc aho 
'"gaged in dcMsing methods appropriate to nonparametne data (5 U) 
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The solutions obtained are not untque, the factors that are dis 
covered are a function of the hypotheses of the investigator (3) The 
factors found may be due to anything which introduces correlation 
between variables and this may be a common level of difficulty 
rather than a fundamental process of some kind (4) Factors identi 
F ■conditions of the 

solve tl r Ft t" ™“'’°'*s, cannot 

ractrd:cre eTT;:e::,:^r,r;''-““ t 

DYNAXf..^ discussed in the section on validity 

section on functionarumv !" "’"'oductory lemarks to the 
three possible meaninoi suggested that the phrase has 

pendence and dependence variation, dynamic inierde 

effect relation) The terhn, ^ process on the other (cause and 
far all involve the demnnisf 'vhich have been described thus 
tion should be raised at concomitant variation A ques 

procedures for establishing of such 

which are of interest tn «£■ i ^ *ypcs of functional organization 
of every science are definpd^ Psychology The complex constructs 
relationships between identifiable!^ dynamic and dependent 
of relationships the atom i v events Force is a set 

An attitude is a comolex nf 1 * ® system of relationships 

as related to an object or tn ^"d affective processes seen 

1*0 described only m term«; nP ” ‘•riage, and cohesiveness can 
actual dynamics of the ® 

are nonetheless miphed system are rarely known but 

Two problems will w 

wnt variation may be expcclS V Although concomi 

dependent or where one fsTd -'■ere events are inter 

correla, = « determ.nant of the other, the fact of 


the dynamic or 
variables (2) A 


correlation is not m u!e?f , m «• 

dependent character of the ref"'"' 
distinction should be made J f '“"“'"Ps between 
“ons of interindividual or int'I^" “"''■'■“■■'ns based on obs'em 
in ivi ual or intragroup variation '■delation as against intra 
even« be shown to cho^, <»>ly m the latTer case can 
The distinction between 

dependence, and causal relattomhinf ''■“■‘■“■■’n- dynamic inter 
familiar one The existence 
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of correlation merely indicates that or more events c ang 
together It cannot be inferred from correlational evi ence a o 
that variables are interrelated dynamically m the vvay tiat ^ 
part of the surface of a bubble depends on every ot er 
maintenance of the integrity of the whole, or that one event ^P^” 
on the others in the sense that raising the temperature o ^8 
increases its volume Projection, for example, is define conceptua 
as a relationship between a person s perception of hosti ity m 
and the presence of repressed liostihty in the perceiver nvestiga i 
would presumably reveal correlation between these two variables 
This evidence, however, would fall short of providing a satisfactory 
demonstration that repressed hostility in the perceiver an P^^'-^P 
t.on of hosuhty m others are dynam.cally or dependently related 
and that there is in tact a process, project.on. trhich can be defi e 
by this relationship In the absence of controls which would rule out 
common determinants of these processes and make possible their 
manipulation, their actual interdependence must e reg 

'‘’'‘’“^ovirnd and h.s collaborators (29 pp 72 73) have discussed to 
problem, noting that the independent manipulation of P™«ssea 
ts often difficult tf not impossible This is true aspecaUy rvhen the 
variables involved are constructs that must e m err j 
effects These authors were concerned with the question o 
mg whether soldiers motivation to go overseas was a ec y 
attitudes toward the British Specifically, ‘he question is i y 
movies depicting the British role m the war and find that both 
mottvation to go overseas and attitude toward the British change, 
■s there any way to find out whether change in a‘t-‘ude has been 
responsible for change in motivation Since correlatio 
'ariables is the onty evidence, it is not clear whether motivation h 
been changed by the movies directly or b, attitude change "oc 
n It clear whether attitude affects motivation or . 

attitude, or whether they interact To establish such r P . 

control of the independent variable is necessary, and m this a e 
impossible In the absence of such controls, "c “a" I ,n „5 
about the nature of the relationships The problem is u q 
and familiar, but often overlooked 

Another familiar problem must be mentioned How far ran we 
go toward establishing dynamic systems in the absence of a 
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quate metrics, one which is appropriate to both the data and the 
t eory of social science? Devising an appropriate metrics becomes 
particularly difficult when the theory and data of a science make 
emands As already indicated, psychological data do not 
fhpnr ^ assumptions of ratio scales and yet many 

on mnrM particularly motivational ones, are based 

Thirh ^ n 1 '''T' and amount 

sutmute “‘i "''■.ch are dispelled in 

and devise describe such constructs accurately 

sTons tiThave measuremen., dec/ 

relationship bctween'^th” comn^^"*''"® character of the 

such complex constructs T^ "“' P™“^'“"'’Ose relations define 
displacement and other Friudian m T’'"’' “"“P" “ 

und opposition of vectors in l T “F *= convergence 

tabu strength, drive strenmh ,!rT" increments of 

Hull model The nroblem * ’ "** action potential in the 

that the crude and mri unsolved, and it can be said only 

present state of in the 

thing approaching a dynamics ofh”?““ 

forces and enerev bui xv.* ^ ^ behavior We talk about vectors, 

When correlations rK"" T 

tion of two or more nror^l denved from the observa 

tional evidence of oreanizati* s number of individuals the correla 
nature of the case, fnlv ‘ Ptn^esses reflects, in the very 

sample of events, such as on * relatedness of a cross sectional 
picture If procedures are to beVnT^i "" ^ ^ luotion 

unities such as motivation and dr ' dealingwith functional 

ton processes (displacement altitudinal structure, substitu 

of smi"'^°'^'r'^"'“’ ““ctttion’miist'be ®™“P cohesiveness 
tc relalions such as are rv 1 tnore than a pattern 

or iniergroup variation The narr* ^ uti interindividual 

as they interact and change twed P““"" “ust be observed 

'he same person or in the ,Ze ""tans observing change 

Jn other ivords .he . ®'°‘'P 
“utphng individuals or grouns’’., '?""'P"'ate variables through 

law by measuring resistance, no, em., "IT"' establishing Ohms 

in onr circuit and at a number of eurrent at one value level 

of additional circuits The altemai.v!'"^ number 
*s. o course, to observe change 
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in the same circuit, or the same individual, or the same group. The 
two techniques should yield the same results if all the lelevant fac 
tors that might affect the relation between variables are the same 
from circuit to circuit, or person to person, or group to group, but 
this is unlikely to be the case, particularly in the study of organisms 
or societies. This problem can be handled only by the study o 
organization and change within the individual, where, for example, 
the interdependent alteration in various aspects of an attitude may 
be observed, despite the fact that the conditions of the change are 
very different from person to person. 

Many psychologists have, of course, been concerned with the 
observation of individual dynamics. Clinicians and other students 
of personality often use this approach exclusively, and more recent y 
attempts have been made to apply factor analysis to observations 
of variation within the single individual.** Cattell suggests that data 
for this purpose may be obtained by observing responses of the same 
individual to the same tests under conditions which change either 
in accordance with some systematic plan or simply under the influ- 
ence of the uncontrolled factors in the situation (F technique) (8, 
9). Presumably any of the procedures discussed thus far could be 
adapted to such data to discover which of the reactions of an 
individual (to items, tests, etc.) show concomitant change, common 
factors, or unidimensionality, but the practical difficulties are ^eat. 

Similar problems arise in the attempt to discover functiona 
unities among group phenomena, though very little systematic 
attention has been given to the matter . For example, group char 
acteristics such as cooperativeness, low intragroup aggressiveness, the 
perception of uniform attitudes, the absence of cliques, and hig i 
output have been regarded as indices of group "cohesiveness.” Sets 
of variables of this sort have been manipulated experimentally and 
observed to affect intragroup behavior (21), but there is little evi- 
dence that such variables change together, so that we are justified 
in identifying a functionally unitary construct to be called cohesive- 
ness. If this were done, it would be analogous to studying organiza- 
tion within the individual. On the other hand, the correlation of 
characteristics between groups is comparable to looking for func- 
tional unity in the covariation of characteristics as you move from 

*iSce the description of other possible patterns of \ariation (8. p. 9G: 9. 
PP- 28/7.), 
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individual to individual Thus, comparative studies of cultures, such 
as Murdock’s correlation of kinship systems and incest taboos (34), 
take the relatively static approach of showing that, as cultures are 
now constituted, some characteristics go together Here, again, it 
may be questioned whether a dynamics can be established from 
these sorts of data 


The Problem of T uncttonal Unities 

those which have been discussed in this section are 

can in attempt is made to discover processes which 

" ongoing 

mon set of idenfi^hf ^ Purposes of scientific observation Any com 

and isolating events buuhTunT‘7 classifying 

by techniques for ^ tliscussed here is that discovered 

pcndence or dependence°ore”'"'® concomitant variation, interdc 

■ng, mtenest correlation lZo‘r °‘’'Z 

dynamic relations ' analysis, and demonstration of 

psychological unities methods of analysis that 

ever hat°e such " 1 “ L"'’"""’'' or that L shall 

do not bleak out cleanly or suMv'' P"'" psychological reality 
organization chance with ti the foci and character of 

sponses are elicited-j e w 'onditions under which re 

asked, the perceptions hv ^ 'ostructions given, the questions 
■IV' siluaiion, ih?. auuJdL TV 'he meaning of 

'h" O'ganizalion orS. I" other words, 

“nines renected in resMn,el?V ""d ‘he functional 

delimited segments, hke atoms 1 '''““ght of not as rigid, clearly 

nT"’’®' ''"S'vble events mor ’ ''‘"•“'e “s shifting and 

nmd medium or ciirrenis 'm , Z “'“"T'" patterns in a 

on I e patterns that can arise nZZu* "^he medium sets limits 
that can be generated, but a h„., V’ "“hdi'y, and on the forces 
medium also helps determine T I “"“"S«>c.es external to the 
magmtude of the resultant forces ‘he Patterns and the 

T^he piiterns re\ealed alsr. 

mg functional unity, wh.cl, do nm "“ethods of determiit 

men,, of psvcholog.ea, realt.v Re,”onTTT^''‘ 

o Thurstone scales often 
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break uno multiple [actors when factor analysed (20), 'nore 
than one factor emerges from tests scaled by the ^ J* 

(40, p 201) Test items chosen for their high indices o is 
lion do not necessarily scale, and a correlation matrix may 
many ways It is important to discover tvhen different me Ms 
yield the same results, but the fact that they sometimes fat to do 
so does not necessarily discredit any procedure Other things being 
equal, the decision about -which method is best must 
scientific fruitfulness of the constructs produced This problem 

be discussed in the next section cmnlv 

Generally, the methods of discovering functional 
reveal the presence of concomitant variation oreov ’ 
proportion of these investigations involves ana yses o 
between individuals It is suggested that the 
interrelations requires the observation of change wit in i 
ual or group rather than interind.vidual or intergroup var at on 
But the statistics of correlation alone are not -^equate to deal with 
problems oE dynamic organization, even when app le o 
the single individual The definition of such dynamic c 
requires the demonstration of actual interdepen ence a 
Uon between part processes and a statement of t e na , 

relationships Establishment of such relations ips nuestion 

experimentaf manipulation and control of the varia e ^ j 
and a metrics that is appropriate to the data and to the theory of 

Thriscusston has dealt largely with 

processes because the problem of functional unity las ^ i 

most extensively in this area But questions about unctiona . 

'vhich are the fundamental constructs of our t m mg m 
answered for all areas of social psychology, including interactional 
processes group structure and related topics 


THE interpretation OF FUNCTIONAL 
UNITIES VALIDITY 

It is obviously not sufficient to stop with the 
nonal unities The processes isolated must be interpreted and given 
meaning if they are to have use in a scientific system 



Methods of Data Coltecfion 


problem of validity Although the distinction between functional 
unity and validity seems clear enough when so stated, it is not, in 
fact, precise In both cases we are concerned with relationships be 
tween events, and many of the problems involved in the search for 
units nre the same as those in establishing validity Furthermore 
some o t le procedures which we have identified as appropriate for 
°"Sanized processes have often been regarded as 
sistenrv example, the demonstration of internal con 

\ali(!itv ^est correlations has been called an index of item 

other validated by correlation with 

as evidence of ^ methods have been discussed m this paper 
comes even notion of dynamic organization 

ity studies The ^ ^ ^ eomplex of interrelations involved in valid 

uy studies The distinction ,s then, not dear cut 

unction betwJe^ f JncTion^ unimportance of the dis 

for our purposes convenient 

ness of process svhateve aspects of the question of related 

IS the question of tsolatmff'^rf'^^ ernployed to identify them One 
and investigation The niL^ wtth sufficient integrity for study 
meaning by findin? the ^ problem of giving these parts 
of events The first oue-:!^ they fit into the whole pattern 

of simpler (though not necpl'! toward the discovery 

raises issues about the p^pfip T ^ segments The second 

the two aspects must bf np?i of the science Ultimately 

suggested, the choice of Tr I for. as sve have already 

made in terms of over all the investigation should be 

"dl depend m part on th#. nature of integrated theory 

"T pen. “I"; “7'"’™" "-h'ch are .dem.fied Mean 
or^nized and more or less on indices are used to indicate 

o functional unity ^Vhen hn ^ P'’ocess, they are regarded as tests 
o tests or molar segment is ^ S*ven item or test or duster 

iiems or processes are considere?i""*‘ organization, other tests or 
die complete neiuork of relaiin ? ^^'^ose discrete To discos er 

abics external to it is to g.^e ^ “"y variable to other van 

S'o.tmean,nB.toexplore Its validity 

Face I'ahdiiy 

It IS possible to name and 

functional unity simplj m terms of obsersed process or 

manifest similarities found in 
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the sttuations or respoftses which have been shown to be {unctionalh 
related This may be called face validity It will be recalled, for exam 
pie, that Guttman proposes that the universe measured by a scale be 

known by the content of the scaled items (40 pp 53 54) This usually 
means that the referent of the attitude which is the common object 
of all the Items is taken as the defining concept If the investiga o 
makes up a test consisting of items about Russia, the instrumen , 

It scales, IS quite arbitrarily called a measure of attitude toward 
Russia When May and Hartshorne (26) devised situations in which 
children could steal or lie or cheat, these concepts were e me 
denotatively The behavior observed in the situation was what was 
meant by stealing, lying, and cheating If the behaviors a een 
shown to be functionally unitary, they would have been interpre 
by their common characteristic, deceit 

The interpretation of the factors discovered by factor analys 
may be made in similar fashion Thnrstone, using the simp e 
ture principle in order to arrne at psychologically meaningful 
factors, has this to say 

In order to interpret the prniniy Iictors it is usually neces 
sary to examine the tests which Insc high saluralion on a facor 
and to dtscoier what ts common to them M'hcn such >’1P" 
esis ha, been found it must be checked by ex iminiug all of c 
tests which have zero or nearly smishing saturations on . 
factor When a factor is hirly well understood then ns 

presence or absence can be predicted with some confidence in a 
new test which has not hitherto been inscst gated factor, ally ( - 

P 7) 

Thus, if a reference axis luns through a group of tests manifesth 
ms oh mg verbal skills, this suggests lint the common factor i 
'crbil factor , , ^,,,i 

There can be no objection to obscrting cermn emio 

that the) are what wc shall mem b> honesty or ” 

or aggrcssucncss, provided as e bear in mind i!ic speci ic an .. 

meaning of the i\ords But certain ciutions must be notet 

often than not, additional meanings arc smuggled in anci ^ 

'“mption IS made that the obscrsaiions arc, m fact, intcrp 

as a sample of a knoun unncnsc-llni. in other words, the insmi 

mcni in question measures some process knoun to mam cs 

■n a specified universe of behaviors under n spccifictl universe ol 
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problem of validity Although the distinction between functional 
unity and validity seems clear enough when so stated, it is not, in 
fact, precise In both cases we are concerned with relationships be 
tween events, and many of the problems involved in the search for 
units are the same as those in establishing validity Furthermore 
some of the procedures which we have identified as appropriate for 
he discovery of organired processes have often been regarded as 
sis^ten” example, the demonstration of internal con 

ahdnv ^ has been called an index of item 

otto ''“'■dated by correlation wtth 

as evidenrp^ F ^ have been discussed in this paper 

00^61 even 1 ‘"’"V The notion of dynamic organization 

tty studies "" ' ' interrelations involved in valid 

A r., , ‘"“'™ “• ‘hen, not clear cut 

tinctton betvTe'eV funcTioMl™''' ‘‘'’‘'”P°''‘anee of the dis 

fur our pur mses ''“’“''‘y. but it is conven.eni 

ness of process, whateveMemTar“’’“‘i °‘ «'a“d 

ts the question of employed to identify them One 

and miesiigation THp sufficient integrity for study 

meaning b ^findlThP " g>v>ng Ihese parts 

of events The first oupcf ^ '^hich they fit into the whole pattern 
of simpler (ihouch no^ toward the discovery 

raises issues about the frenT^J* ^ irreducible) segments The second 
the two aspects must bf nearly f Ultimately 

suggested, the choice of na » ^ egraied, for, as we have already 
made in terms of over all the for investigation should be 

'wU depend in part on iHp nature of integrated theory 

uiule, our point is simnlv , "'hich are identified Mean 

organized and more or less isn/Vi"^ ^n indices are used to indicate 
of functional unity When ho^ ^ ihey are regarded as tests 

of tests or molar segment » ^ given item or test or cluster 

nems or processes are considerp^^"'* ^I'g^nization, other tests or 
the complete neissork of reU, T discrete To discover 

abics external to it is to ms- , ‘ of any variable to other van 

S-'eumeamng to explore tts vahd.t, 

r ace Validity 

It » posnble to name and ,n„r„ 
functtonal unity simpiv m trnv. *^*^1 any observed process or 
) term, „r s.m.lanttes found ,n 
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the sttuattons or responses whtch have been shown to be 
related This may be called face valid. ty It will be recalled, 
pie that Guttman proposes that the un.verse by a scale be 

Lown by the content of the scaled items (40 pp 53 54) This usi y 
means that the referent of the at.itnde which is the “mmo" objec 
of all the Items is taken as the defining concept If the >nvest.gato 
makes up a test consisting of items about Russia the instrument if 
It scales IS quite arbitrarily called a measure of attitude toward 
Russia When May and Hartshorne (26) devised situations in which 
children could stLl or he or cheat these concepts were defined 
denotatively The behavior observed in the situation was what was 
meant by sLling lying and cheating » I’"* 

shown to^be functionally unitary they would have been interpreted 
by their common characteristic deceit , . r * i 

The interpretation of the factors discovered by factor analysts 
may be made in similar fashion Thiirstone using the --P' «™c 
ture principle m order to arrive at psychologically meaningful 
factors has this to say 

In order to interpret the pr.nn.y 1 iclors it is usually nece, 
sary to examine the tests which hive high saturation on a factor 
Ji to discover what is common to then, M hen such an hypnt i 
ests has been found u must be checked by ex .mining all of il.e 
tests which have «ro or nearly vanishing saturations on the 
factor When a factor is fairly well understood then us 

presence or absence can be predicted with some confidence in , 
'new test which has not hitherto been investigated fac.or.ally (« 

P 7) 

Thus if a reference axis tuns thiough a group of tests manifestly 
mvolt^ verbarsktlls this suggests that the common factor is a 

'"’’TherTcan be no objection to observing certain behaviors and 

vla.ing that they are whai we shall mean by 

° ^ \ip bear in mind ilic specilic and limited 

oraggresstveness P-; , 'Lrlam cannons nmst be noted 

"ob:„"';\:'nm ad mon:; reamngs arc smuggled tn and .he nv 

mrptmrts made that the observauons ■>« ■"/,ror<l’, .IiTm’'”^ 
as a sample of a known univcrsc-that. in other words the mstn, 
men. m qLsnon measures some process '^clf 

tn a specVr.etl nn.vcrsc of behaviors under a spec.fietl universe m 
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conditions Thus a test is assumed to sample adequately and in a 
systematic way the manifestations of some process called attitude 
touard Russia, or aggression or honesty, or cohesiveness However 
reasonable the hypothesis may appear, the character and limits of 
the behavior universe sampled cannot be set merely by identifying 
a common characteristic of the items of a test and assigning to the 
measure the generality possessed by the name of that characteristic 
iiie limits must be determined 

necessary to be certain that the manifest 
troll 1 ^ ^ significant one Since unobserved and uncon 

ven m a n for the order among the .terns 

TcL te asnl S' f be ruled out before 

example, that percrpnoroV.heTr TT " '' 

of the Cuitrmn logical relations between the items 

res;t^w:r 

people want to be lomLl If th ^ ^ ^ 

sider that the order hes m th^ misleading to con 

are present in mtpreungf^a^^^ 

lering aiound an axi? arf u loadings on tests If the tests clus 
volved may well be thp ri- words are in 

be entertained and checked^^Th possibilities should 

common difficulty level wh.rK , >^>gbt, for example, have a 

or they might be less fificn a^*fferent from those of other tests 
and so on One advantace^of fatiguing than other tests 

different sets of data md same factors m many 

m minor ^\ays is that which vary 

accidental bases of correlation may be ruled 


Predtctwn to a Criterion 

instrument is the simple^eni^'^^ determining validity of an 
measures made by the instrument correlation between the 

relation is high, the measure ,/®"r criterion When the cor 
the stated criterion and the test m*. ' u *** Pfcdiclton to 

college aptitude tests or tests of ait t a by the criterion as in 

The reader will recall the disL«.^ ‘o'^^rd going to college 

on of the factors which affect 
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the CMdence o£ funct.onal un.ty between tno 

the weights assigned to test items, the extent to which the va ables 
confom to the assumptions implicit in the statistics sed th 
difficulty level of the tests in conjunction with 

the measured characteristic in the population sample, ‘he election 
of specific mathematical procedures tor expressing the ‘ela 'o^ 
All these problems are equally serious in validating a ^ 

a criterion, and the reader is Onh two further 

more detailed discussion of them (4, pp ^ I , i^rtmn of a 
points will be considered here the problem of the selection of a 
criterion and the limitations of this method of validation 

The choice of a criterion will be determined by P “ “ 
or theoretical consideration The practical need to selec 
salesmen will dictate that some measure of success m *^'“^"*h.p 
be taken as a criterion, or the demand tor information o" how many 
soldiers will go to college after lea\ing the rmy aramst 

requires that a test of attitude toward this topic e va ‘ 
the criterion of actually going to college Again, i e others a 
that lack of self confidence makes people poor ju ges ’ j 

test of self confidence might be validated a^mst 
accuracy in judging others In reality, the se ection J gjj 
IS based on a Theory about the relation of the P^““ 

to the criterion situation or behavior, but 
solved IS some common sense assumption, such as the 
a person who dislikes Negroes will be more likely "> 
segregating them than a person who likes them or the 
that fhe man who has a favorable attitude toward the ^ ^ 

more likely to go to church than the one who expresses bitter resent 

merit against the church , 

It will be suggested tliat this level of theorizing is ‘"^equate 

for the validation of tools or constructs of 

fulness For the present u need be said only that in the absent o 
an integrated theory of behavior which dictates ‘he election of 
validating criteria, literally any behavioml 

chosen if it is itself to be predicted or if it is ju g „„,de 

process of interest Practical common sense has o ten 

Any demonstrated relationship between a ‘he 

data yielded by an instrument provides an additional fragment 
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of meaning, and, as the measures are shown to be related to still 
other criteria, they take on further significance If, for example, a 
scale purporting to measure attitude toward Britain proves to be 
correlated with amount of hisloncal knowledge, with church mem 
bership, and with income, we know more about the scale than we 
woidd if only one of these relationships had been established 
Such information may lead to suggestive hypotheses about the source 
cm *" studies A protest must be 

proliferation of blindly empirical 
the incrpiTem'^ Without the disciplined guidance of theory, for 
correlatinn< ° from the accumulation of miscellaneous 

uses of th “■'mutely approach zero The short run practical 

me of aotitiide 1 ° . ''t " uiu evident m the design 

to have provided the°'kmd"T 

a scientific structure ° ‘""ttpretatton needed m building 

uresIf''thos”‘;mrr.a„?'’'“l''l "> ™I.date meas 

as intervening variables fan 'ohstructs which are known 

tnow that a que!tionnaut ■' is useful to 

differently by those who ar" toward religion is answered 

not, but before sucL in ™ nnd those who are 

other steps are ^ essai Tfh ■’f' significance, 

of the attitude proce7and 

in the determination of hehnv with other processes 

must be made to discover whtlh "*"1' "orked out, and studies 
We turn then, to a discussion f hypotheses are supported 

ducussiou of this concept of validation 

dictions from theory mav^hp^*^ to validation through testing pre 
measured process tsVeu „oi The meaning of any 

used in isolating u from other n?" ^ ^ ^ ‘description of operations 
of quantity but also by knowledJ^^^^^ assigning some index 

and their influence on it Conseo ° ‘ofluence on other processes 
construct and of the defining ^ "“y* establish the validity of a 
iniestigations This iniolves all conduct experimental 

pro lems of formulating theory. 
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deducmg consequences, and testing the deductions under conditions 

of controlled observation. nf 

If behavior theory leads to deductions about conditions of 

change in process rl and the effects of ^ on other 
must be found to determine the accuracy of these deductiom When 
predictions prove to be correct, both the theory and the construct 
L measureZre validated in some degree Where the predictmns are 
seriously in error, it is difhcult to spot the troub e unless euher th 
theory or the method is sufficiently well 

suspZon Thus. If the orbits of a number of *tats and^the theory 

of gravitation all point to the existence o “ “ imperfections 
body, the failure to find the star will be =>“”buted to n^perteamn 

m *e instrument unless the investigator 

be conditions under Ihe mhe^r hand, both measures 

longer tell the whole story If. on th 

and theories are suspect, untangling familiar to 

involve the clumsy methods ^(^""i.dation of theory and 

the psychologist In any case, « « ^ simultaneously and 

of instruments of observation tend p 3„umu 

that they can be separated only m so f P theoretical 

lated to suggest ‘^at pted.ctiom 

conversely, that although => ^ ,h,„gs go wrong when 

predictions from theories A, B, 

predictions are made from theory X interpreta 

There are still other complications which maRe t 

non of the unconfirmed Interaction of many 

and position of a star is a predic ^^5 he predicted from 

stellar bodies, so must ^ehavim of P measurement of any 
more than one variable This roe m 

or all of the variables and the *h “7 .t is no wonder 

behavior may be in error “"Jn ,he oversimplified and naive 

that validation procedures began variables m the hope 

attempt to predict single cr.tena from 

that by luck some clear cut ,Lu up a score 

Even though the m the prediction, .he 

in favor of the theories and methods i 
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evidence from one such confirmation is of course, never conclusive 
It must be supported by further diecks in situations where the proc 
ess measured in a certain fashion is combined with other variables to 
produce predictions of other consequences By this procedure, the 
limitations of a construct, as defined by a certain method of observa 
tion, will become apparent as evidence accumulates regardinff its 
successes and failures 


I *fi examples among psychological studies 

w uc 1 ustrate these points, because this method of validating 
easures has rarely been used deliberately, and never extensively 
aspects o the California studies of the Authoritarian Per 
was clarify the problem (2) The study 

as attitiirlpl^t theory that such psychological processes 

discipline outgroups toward authority figures toward 

but that the naf conventional morals are not independent 
some chartter^ ^ is a function of 

varying from ^ adjusting to conflict and hostility, 

to extremes of rpnv^ ^ tnsightful, direct, and ego integrated attack 
mechanisms It is 

will orodure latter pattern of adjustment 

out groups accentanr,. t u ** attitudes of tolerance for 

evaluation’ofaufhomy ^ 

between the phenotvn P^’csented involves correlations 

cor.eIa..ons a^Zhlv'ZT ^he 

■s tnie and the rneasure'mpnt''^ would be expected if the theory 
that this study does not valid However, it is obvious 

of the hypotheses o ":r ’"’““y satisfactory test either 

«>™pt to test pted.ct.„„:rf“^“ since there is no 

control the variables The 1 conditions that systematically 

tional techniques in the limitations of correla 

discussed on pages 278 ^8^ ^ ° yuamic organization, which svere 
Stesens (39, pp j,_, 

nient in salidating measures on the present predica 

that we are now dealing largL ‘ror'";"® suggesting 

reined by unlnown laws to the Lvch'i '"tfuttints.’ 

i\e are really interested We ob ^ ^ “omgical dimensions in which 

we observe verbal statements and'lnfcra,?!"?" 

aiiitutie He goes on to say. 



Problems of Objective Observation 


291 


The difference, then, bct^secn an indicant and a measure is 
just this the indicant is a presumed effect or correlate bearing 
an unknown (but usually monotonic) rchtion to some under 
lying phenomenon, whereas a mtisurt is a scaled \ahie of the 
phenomenon itself Indicants htve the adv image of con\emence 
Measures have the advaniige oi validity 

This may seem to suggest a somewhat more clear cut definition of 
validity than has been proposed here, by restricting it to the rela 
tionship between the measurement and the process measured This 
ts what we would really like to know, and it would be convenient 
if ways could be found to discover this relation without recourse 
to the complicated procedures of predicting from the measures 
and testing the predictions The trouble is that there is no direct 
access to the underlying phenomena It appears that we shall always 
observe indicants, for we cannot get inside and watch the attitude 
at work The hope is that we shall approximate more and more 
closely the law which relates indicant and the thing we want to 
nieasure That we have done so can be known only from the observa 
tion that if we assume some specific relation of process and measure, 
our predictions to other events are more accurate than when some 
other relation is postulated The relationship must be guessed it and 
tested by us fruits 

We return briefly to a point which was mentioned in the eirly 
pages of this chapter When evidence of the validity of an instrii 
ment has been obtained, it must be remembered that this validity 
has been demonstrated under a circumscribed set of conditions 
These conditions may be explicit or implicit, their range ma) oi 
niay not be carefully spelled out, but they are, in any case, a neces 
S3^ry part of the description of the validity (or functional unity or 
reliability) of any instrument, for u is unlikely that any relationship 
IS ever completely and unexceptionally true There are limits within 
which it holds, and evidence should be provided about the condi 
tions which must be constant and those that may vary without 
disrupting the prediction on which the evidence for validity rests 
For example, it would be questioned whether the findings of the 
California study could be reproduced in some other culture, whether 
they hold for all motiv-ational conditions, for all levels of mielli 
gence If they do not hold up, the scales are inappropriate for these 
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different populations or the relationships between the processes 
measured are different under the new conditions The discovery of 
methods of isolating and controlling the relevant variables which 
are responsible for correlations and good predictions is the basic 
methodological problem 


STABILITY OF MEASURES RELIABILITY 

elahoraT concept of reliability does not require extended 

a d'Zr T,. texts to 

considered n'" ^5) Two questions will be 
itv and W "/J" (') '*'0 meaning of reliabil 

(2) What are th ' 1 from functional unity and validity? 

to refer to a functiona"ii''°™ '’^en used in this discussion 

properly identified m ^ ‘he notion that such units, when 

pcedtcltonrio p ichZTcIf • '““'“'o' 

■mportant quesUonZT Reliability raises the further 

observed, measured, and rfeZl'’'w'h '' 

the reliability of their nh. * “ investigators inquire into 

"ell they can coZ "re asking themselves how 

peated attempts to re establish™"'* *"0^ "^esponse on re 

It is a basic scientiRr- observational situation 
■he results “oHuons are constant, 

there must have been lomT*^ /Consequently, if the results change, 
sponse or inferred const ** When some re 

specified set of conditions'^^ k predictable from a 

unstable, reliabthty is losv* when predictions are 

The sources of unreliabiluy w,I| „r 
to situation Loevmcer h ^ **' vary from situation 

she calls content factors and t''* types of error which 

content factors are found m variations m efficiency The 

in the instigating conditions LeZ"'''"''’ ■'■■PO"m-ie, 

source of error m a uay that mat actually interprets this 

rather than of unreliabihtv m ^ ^ source of heterogeneity 

locates the problem in ilie lict ri...r leaning of the term, for she 
some of the test items involve 
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different abilities from those required by the test as a whole But 
when the meaning of unreliability is taken as mstabdity of ■‘“ctio . 
the character of the items introduces unrehab.hty in so far as they 
are not sufficiently specific in their effects to tap the 
or instigate the same reaction on different " 

second type of error refers to the numerous 

in the individual that inffuence reliability: f 

boredom, and similar variables which change from one testing to 

Two other types of variable should be added -hich ™y -odify 
the measured results from one observation to the next: (1) 
ing social and physical stimuli which affect the reaction. P) ™ 
atfons in the r^ording and interpretation of the b^avioral events^ 
This latter source of variable error may be isolated from o'he 
by the familiar procedure of ascertaining coder or 
bility through independent observations and 

of tL protocols by several persons. The elimination of the other 
causes of variation that lie in items, in the total “at om and in 
the individual requires the application of the usual techniques 

“.-Sr;.” . J r.— 

the reliability or unreliability is a property on y o whole 

itself, for the error observed is the result of -nation m the who^ 
complex of determinants of the measured event, p 
produces stable measures under one set of conditions or 
l^ample, or with one person may not do so -“h “^her condmon^^ 
with other samples, or with another indivi ua . si j-giiably 

outset, any statement about what a test measures and h™ cel.a y 
the measurement is made must be ac“'“P“‘'='' „^V<rur 
regarding the conditions under which t e s a em j.terminine 
It is necessary to examine briefly the opc-nons or^d— 
reliability in order to discover their re ation procedures, 

tion of the term and the attendant P™s“retest, 

The methods which involve correlation the j the 

odd^ven. or split-half and poshfom wUh 

stability with which 'nd‘''“f"“}^ or on different 

respect to one another on repetition o 
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sets of Items which are assumed to measure the same process “ The 
re la ility of means, of differences and of other statistical indices 
attempts to estimate, in accordance with the theory of error, the 
probability that the true value of the index (viz, a mean obtained 
irom an infinite number of measures of the same universe) will fall 
within a certain range of values As Loevinger points out, 

IVo matter how the reliability coefficient is computed the 
sties of reliability assume that (a) the variable error factor has 
n expected or average value of zero (b) the error factor in one 
too tamed scores is uncorrelated with that in another set 
er similar the test may be (e) the error factor in a set of 
nf ill with the true scores and (d) the variances 

e error factors in two comparable tests are equal (32 p 6} 

■£ determinants of the 
tion to the next systematic fashion from one observa 

measurement affect th^ ^ items, if residues of the first 

or sets changed m ®^cond, as when answers are remembered 
other syStfe nesting, or if any 

test And if the ro *” conditions occur from test to 

indices give an imr/-? *^^*^*^* «nnoi be fulfilled, then reliability 
to ttiuch measures maTL^'c'’ tnd.catior of the extent 

Condition ^ expected to fluctuate under the given 

tempts to follow the lo f retest method which at 

same conditions (same ^ cephcating observations under the 
that the measures on succ*^*' * 2 me persons), runs into the difficulty 
that the variation from lesH**'^^ occasions are not independent, and 
On the other hand altern° 

the problem of how to senarat^ n*™* procedures pose 

from unreliability When ^ ^^®cts of heterogeneity of process 

devised the attempt is mad"^° more forms of a test are being 
to involve the same process ‘^^"^‘cuct parallel items that seem 
assumed to be an index of * fbeir correlation is often 

houever this solution is un!^f,?f Loevinger has insisted, 

'-See j** ®ctory It u necessary to provide 

nu.nionc Spearman llroivn^ of the methods of Kelley 

and Richardson and others 
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more objective evidence than this that the items <5“; 

the same process But i£ intercorrelation between ‘h" 

as an index of reliability, " X the'co'ebtions 

:L“trf r r^uments are in™ 

the same process or the same set of processes and also 

measunng them reliably But if the correlation is loj^ « canno b 

clear whether this is because of the instability of ”Xe " 

because the different sets of items are m fact 

things Our present techniques do not make possi e p 

suggested that reliability be 

thought of as rXing to the amount of 'f;* X“ure 

or observation reveal when repeated under con i i 

that only random variable errors affect this ^ 

measure or observation is reliable does not 

a signiHcant variable is being measured or ^ 

measure or one that is uncontaminated by irre 

These are the problems of validity and homogeneity TyXms 

a measure is reliable means simply that the ‘“P" m the 

of the measured event-the instigating stimuli. ‘hy“ure" for 

reacting individual, observational techniques 

handling the observations and reducing them "> ^e 

are all Lffiaently under control for us to be able to reproduce 

results within stated limits rpnrp« 5 pnts 

Whether an operation can be found which ^ccu L only 
the degree of stability of measures under con i i 
to varSble error turns on the problem of d-=vsmg ways 

eating observations so that they will be indepen e 
Meanwhile m spite of the ambiguity of the indices which are use 
for this purpose, the justification for their con inu 
even tho^- may be unable to estimate ely the vanaWe 

error on a test we can say that low correlations e w nst,£y 

a retest indicate that conditions are not sufficien y j 

using the particular method until the ° correlation 

be identifi’ed and controlled On the other “and. ■£ he coire^ 

IS high the situation is a stable one that warra j,^ ,n 

non in order to find out whether the source of “>= 

some artifact such as mere recall and repetition of a previous 
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sponse, or m having found a way to control the determinants of 
an important variable which is the object of study 


CONCLUSION 

chapter has reviewed certain problems of objective obser 
with Pin" ^ methods for dealing with these nroblems, 

^n haf h ‘ ' “d limitations Aden 

validity, ard"Siht 7 ‘"“'“‘’' “ “"“y’ 

evaluated problems must be seen and 
'Snc?scie„ e h, “ “,1“"' — pl.ons and methods of 

about the relationslf'' "'h ^ ’’“''‘’'"S comprehensive hypotheses 

■f these ^“nfarp'r'" 

the predicted consequences'^^ d"*' methods of observing 

vation To delimit and d g conditions of controlled obser 

terpret their meaning (vaIid'!tV°'’“d"“'“ ""“y)’ “ 

stability (rehabilitvi E, 1 produce evidence of their 

logic The design of oh™ ™ wocking within the framework of this 
therefore, a theory abojri'h™ '"^'^fotts and procedures requires, 
variable to be measured 

tlcpondencies, anT .., *“ “''"nttnants, Its dynamic inter 

unity, validity and evidences of functional 

evidence, subject to the i* obtained are, like all scientific 

the observations, for the imposed by the conditions of 

tional procedures are mr., eharacleristics of the observa 

are contingent on those conditions 
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CHAPTER SEVEN 


The Use of Documents, 
Records, Census Materials, 
and Indices 

Robeit C Angell and Ronald Freedman 


those treated in'mhe "’”**'’* treated in tins chapter differ front 
materials and md ^ m that documents records census 

scientist in a forni over' Jhiirhrh ''''' 

trast when the social nsrch '''^'tltitely little control In con 

either participant or n ^ ° ‘^**1 uses the method of obsersation 
of the behasior of can focus on those aspect*- 

interest for him if he population that have theoretical 

or frame the insirumentT^ ^oestionnaires he can choose 

the subject can be guided°b'"'i'"* ‘"tc''' '™"’S 

are unclear the interviewer ral interviewer and if crucial jioinis 
Although there are excenti 'he matter is elucidated 

normally come to the invesncator tceords. and indices 

either a participant in a social s.t “me other person 

of a system ol recording or the rre« originator 
tor of an index determined the 
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form of the data These materials frequently have to be recast in 
terms of the research problem in hand before they are fully usable 

by the scientist , . , 

There is perhaps another feature of these research methods 
that distinguishes them Because the data come ready made, they 
do not depend upon the reach of a specific investigator or research 
team, whereas data that are obtained through scientiHc observation, 
through tests and questionnaires, and through interviesss are gath 
ered for a specific purpose characteristic of a particular researc 
design and are drasvn only from universes in space and ° 

which the formulators of that design can send investigators Docu 
ments, records, and indices, on the other hand, may bring together 
data for scientific analysis from remote times and places 


DOCUMENTS 


Since we are here concerned with research techniques pertinent 
to social psychology, the documents that wil e mpnt mav 

that gm insight into processes of interaction A document may 
descrfbe a process of personal or group developmen A only 
limitation on the complexity of the situations dealt > ‘hm 

Its writer must be able to embrace the situations adequately in h« 
thought and treatment The scientist must find in the “«ou P 
the facts he needs to perform a theoretically satis ying termed 

The category of documents thus delineated may be ^™ed 
expressive documents They are at one end of a 

other end of svhich are such documents as court --^jds. official 
histones and proceedings of commissions In etween 
as newspaper stories and memoirs of diplomats -b.ch mav g-v- 
humanLm data but which rarely yield 
embracing statements of the interactive processes p 
tnents have been discussed at length in „„„„ 

the Social Science Research Council (3 7, ) K.himtTraDhies 

graphs contained in these bulletins gne extensive bibliographies 

(3. pp 192 201,26, pp 164 75) 
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Types of Documents 

We shall divide expressive documents into (a) personal letters 
(b) life histones either diaries autobiographies or what Abel (2) 
calls biograms (see p 304) and (c) accounts of small group process 
As will appear in the discussions of the separate categories expres 
sive documents are not always spontaneous for their production 
may sometimes be stimulated by the scientific in\estigator This 
does not mean however that he necessarily imposes his definition 
0 t e situation upon the writer If the insestigator really wants a 
tree and untrammeled treatment of the subject lie will refrain from 
structuring u except for indicating clearly the area of life experience 
he wants covered * 


Letters 

exnresswp^H^ betters constitute the most frequently asailable type of 
STan on™'"" leaned upon Them 

eZnUnZ r ■" monumental Th, PoUsh 

both bv Thom?^ “"u (52) Senes of letters were collected 

60 ser. s ,64 Znanteckt tn Poland 

used matnl J r ^P^Ouced tn the text They are 

mary groups wrouX changes in Polish peasant pri 

peasants and emigrationTM“*'T‘'‘"“‘'°" P"°’u'“rization of the 
his critique of thi« i, members Although Blumer m 

authors previous knnli^ri letters supplement the 

way round he concedes of Polish society rather than the other 
great deal and derived r mulled over the letters a 

gestions and generali 7 at,o*^ c"* 

theoretical statements (7 incorporated into their 

of data so extensively ^ P 'O other study has used this type 
The value of letters as e 

culture in which the writ documents ivill vary with the 

example that personal corrcm^ ™mursed ft is probably true for 
of life situations m pioneer?^ revelatory 

was neither the competition of?,!!!!! 

tunnies to communicate bv othn- '™’' interests nor the oppor 

us of the art of letter writine o??”!'*"" 

g Only the separations of svar have 
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in our day, revived somewliat the impulse to communicate ones 
inner life freely and fully in personal letters. 


Life Histories 

Diaries are a second type of expressive documents. AUlmugh 
they have been used extensively by histortans and teen r^ 

ommended by an eminent psychologist as "the 
par excellence" (3. p. 95), they have not been used much m soctah 
psychological research. Two examples in attemp s are mad 

to formulate hypotheses, in part at least from dtar.es, are Runner s 
study of social distance in adolescence (40) an /*'' 
suicide (11). There seems to ‘te v^ 

dra^vn upon large numbers ot diaries. y ;nv<»stirate 

difficult to collect enough of them at the adult 

nomothetically a an^sci'emiBc hypothesis 

develop scientific generalizations. Aim y social 

would^retiuire the fmposition of «,ntroh 

dass. and this would greatly limit th ^ stimulate 

could obtain the diaries. It is conceiva ^-^pA *0 investigate, 

the keeping of diaries by the g-^P^’^jrmtmst m va; s^m'^rk: 
but there would be a strong lenden^ Vtitificallv comparable, 
edly that the documents would not be ot pr/chological 

\llport, in The Use of Personal 

Science (3), argues points out that better pre- 

including diaries, in scientific svork. P freouently be made 
dictions of the behavior of specific P'™- '^Xm generaliza- 
by careful study of their past than by "PP^S ^ ® le. How- 

tions drawn from the study of P”P“ “^^,.n[s may be for the psy- 

ever useful such idiographic use o unbearable burden on 

chological therapist, it would 

social psychology if It were adopte “ idiographic "laws" 

knowledge. It may be person^ but also for a few 

of behavior not only for maladj P p^jon. The general 

outstanding leaders, such as Stalin ° neralizations that are 

advance, however, must come fro nder specific conditions, 

applicable to w-hole categories P“P give much promise 

If diaries are not kept frequen } S 
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of scientific usefulness, the case is even more discouraging with spon- 
taneous autobiographies, and really perceptive ones from a social- 
psychological point of view are rare indeed. H. G. Wells wrote one of 
the few (62). Thomas and Znaniecki had real success in commission- 
^ immigrant to America (52. Vol. II, pp. 

6). This document goes into minute detail about almost 


every aspect of the writers life. Clifford R. Shaw has likewise ob- 
histories from delinquent boys (43, 44, 45). 
flrs. include in the present category documents that 

hp -^ 1 ? telling his autobiography. This is not to 

,inc= in .uch an inter- 
life lhaf ent is stimulated to deal with those aspects of his 

anthronof*^^' * eoretically significant for the investigator. Cultural 
“phte trvl ‘'''= of ^pSen autobiog- 

it as the "nas • ° ttnalysis of this technique, characterizes 

appropriatrtoTn He states that it is more 

interview is bettersuihedT’’'’^,!”"® whereas the active 

spoken au tohiovrl T- ° ‘^fhure as such. Notable 

-d 

nomothetic inThp*I*f documents has hardly ever been 

to gather data thar t investigator has wanted either 
life-DoIIard's criteria^ enable him to understand the individual 
become cenerallv ar ^“‘gned for this purpose (I8)-or to 
Winnebafo subculture. The 

biography of a y obtained by Radin (37) and the auto- 
good examples of the by Sutherland (16) are 

future full autobiographies'^ k *" 

to control variables and obtained from enough people 

probably continue to ^ i^ypoiheses. Such documents svill 
giving rise to "hunchp^'’ ^ nomothetic research mainly by 

Another approach is ? hypotheses, 

lecting svhat he calls biogramTJpx^i!*-^ prescription for col- 

hfe records written, at the ' ^ coined to cover 

large number of persons social scientist, by a 

periences. Chalasinski used this involved in similar ex- 

Young Peasant Generation in Pow"/'?«v^ preparing his The 
It for his Why Hitler Came Into P ^ himself used 

research problem was to determ 5 « former case the 

e the role and significance of 
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youth organizations in peasant life, and in the latter it was to d.s 
cover the factors leading to participation in the National Socialist 
movement In both cases the documents were obtained by setting 
up a prize contest which offered a large number of awards Ano 
nymity was assured Those who were eligible and the area of life 
to be covered were carefully specified in the announcements A e 
believes that this method provides freely written mass data on 
specific types of experience suitable for analysis by the social scientist 
A somewhat similar technique was used in a study of German refu 
gees from Hitler (4) 

Accounts of Small group Process 

Accounts of small group process by a participant are 
written spontaneously that they have not been 
any large investigation In one case, at least, sue ocum 
been sLuIated by the researcher for a nomothetic type study 
Angell obtained from University students 50 f ” ^ 

life before and after the impact of the depression (5) The wr 
were paid a small fee They wrote from a rather ^road outline 
which suggested aspects of family life that were to f-milv’s 

sample topic was "Discuss the external conditions of your fam y 
existence prior to the decrease m income Touch 
neighborhood, the house and yard, the family's material 
etc • A number of hypotheses about family organization under 
were tested by analysis of the data 


Use of Documents 

The peculiar value of expressive documents is m them 

life IS discussed in terms meaningful to those involved The pre 
conceptions of the investigator do not determine ‘he nature of the 
data obtained To the degree that social psychology n 
stand the "definition of the situation" of participants, 
tnents constitute an invaluable source of scienti c m oima 
Because expressive documents are rarely 
hy the investigator to afford a crucial test o speci ' ^ ' 

they have generally been used m the exploratory ra „pri,ans 
final stages of the research process Their ^eaten « -^P^j ^,P; 

has been in giving investigators a feel 
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producing hunches ' with respect to the most fruitful way of 
conceptualizing the problem The reseaich scientist must become 
intimately familiar with the situation under study, and one of the 
est ways to do this is through careful reading of insightful expres 
siye documents Thomas and Znaniccki, for instance, conceptualized 
three types of personality-the Philistine, the Bohemian, and the 
documentary materials (52, Vol II. 


documents are capable not only of identifying the 
esesemhnH'^^'^^^v.^^ ^ specific problem but of suggesting hypoth 
influpnrM K ^ Variables These formulations will, of course, be 
and dpdiiff ^ existing theory and thus they are at once inductive 
dlustrates thi! admtaM^ ' of the Conwell document (16) 

hypotheses 'remap scientific process is the verification of 

sufficient to cive Senerahzauons must be confronted with data 

4 "rely 

plishing this but thJ,. as one of the means of accom* 

because they may not 

numbers of caset Tn, exactly the data necessary m sufficient 
provide a better questionnaires can 

can be set up to obt * pertinent situations because they 

precisely relevant many cases the information that is 

stage Just because thevarf., ■ “"’P'utely eliminated at this 
of the investigator the ° ®minated by the conceptual scheme 
obtained by other methLrTh""' excellent check on data 
the srarch for negative evidence appropriate to 

research is beset wth ma **°^"tuuts for purposes of nomothetic 
discuss the following six y. ^ud problems We shall 

means. ( 2 ) the representativei^^TT”^ ^ 

quacy, (4) the rehabihtv of tu documents, (3) their ade 

analysts interpretation of th ^ (5) the reliability of the 

Endings record, and (6) the validity of the 

Experience seems to show th 

specific topic or process is more ^ document focused on some 
that covers all aspects of a nomothetically than one 

u s I e The smaller the area about 



Documents, Records, Census Materials, and Indices 


307 


which the informant is writing, the more likely that all the theo 
retically relevant material will be given in the document This not 
only makes possible the framing of specific hypotheses but gives a 
richness of context that may produce a new conceptual orientation 

Since the investigator is usually trying to arrive at scientific 
statements that have validity beyond the data from which they are 
drawn, he must be concerned with the representativeness of the 
data This poses a well nigh insuperable problem for one who 
would work with expressive documents Even if we allow that the 
possibility of dictation obviates the exclusion of those who cannot 
write, there is still the fact that some people are much more inter 
ested in expressing themselves than others It would take the great 
est ingenuity to set up a random sample from any universe of 
persons such that each member of the sample had produced a 
spontaneous document useful for scientific purposes Even when 
the sample is obtained first and the members of it are then stimu 
lated to produce documents to order—which is the best procedure 
to ensure representativeness— the probability that they will all per 
form the task satisfactorily is very small This is perhaps the main 
reason why expressive documents are thought to be more valuable 
in the exploratory phases of research than in the definitive testing 
of hypotheses 

The question of adequacy is not one that concerns only the 
documents written by the less intelligent Even highly educated 
persons may not see all around a problem They may recount only 
those aspects of it in which they are interested and leave out alto 
gether aspects of great relevance for social psychological analysis 
Moreover, they may not give enough of the background and context 
to make clear the significance of the behavior described The pro 
vision of a broad topical outline is the best method of coping with 
rhis problem, but it can hardly be expected to solve it 

The reliability of the record largely turns upon the truthfulness 
of die informant How can one tell whether the author is truthfuP 
A most interesting finding in this connection is that of Frenkel 
Erunswik persons rated by associates as unreliable are prone to 
superlative and absolute statements and to excessive repetition (24) 
f'ossibly such a criterion would enable the investigator to reject 
unreliable documents 
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A quite different problem is the reliability of the interpretation 
made by the social ps)chologist Hou do ue know that another 
scientist would see the same data in the document? This can be, and 
as been, tested (10, 12, 48) The upshot of these tests is definitely 
encouraging Well trained investigators will agree to a scientifically 
satis actory degree on what the personality traits and attitudes of 
persons are and how they are likely to behave under specified cir 
cumstances In order to convince others that this is true, it is highly 
iicVi .4 ^ t inany expressive documents as possible be pub 
^ investigators can determine to vslnt extent they 

would agree with the interpretations made 

that thf» problem of validity How does one know 

CartwriPht anH v u c^ieck is to use other criteria, as 

compared results*^" thi done The former 

a diary, with actual tecr of cise materials including 

the validity of thf> mr of the subject and found that 

was greater than the rehabduv 

tbis means IS that earhmor, interpretations What 

in certain area? of j particularly sound interpretations 

were no." 

terpreters of document ,s not I among m 

as has usually been thought damaging scientifically 

fn Stouffer s studv f 

Prohibition of 238 wr r rated the attitudes toward 

by correlating them with^th ratings were validated 

The validity coefficient was 0 

that obtained m the Par* i_ ** considerably higher than 

study the reliability French study but in Stouffer s 

reliability and vahity renL?!f k ^ 'Fhe higher 

from the fact that the b ^ ^ Stouffer almost certainlv resulted 

focused on a narrow validating scale were 

wnght and French predicted a“'tudes toward Prohibition Cart 
eral character how the subi ^^pressive documents of a gen 
questions comprising a ner^^^i answer a broad battery of 

that valid inferences can be This would seem to show 

a specific area are adequate 'Th^ documentary materials for 
« perfectly compatible with the 
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view expressed earlier that a focused document gives the social 
psychologist the degree of "density of data" necessarv to the full 
employment of his theoretical formulations 

This general review of the uses of expressive documenis in 
social psychological research shows that they h-ve genuine value 
particularly m the development of concepts and hypotheses but 
that for a complete piece of nomothetic research they need to be 
supplemented by other approaches Several good , 

Thrasher’s The Gang (53), illustrate combinations 
Over a period of seven years Thrasher directly observed 'he Wc "f 
gangs, persuaded youths to write their own stones, 
members, examined social agency and court -“'f 
newspaper accounts of gang activities Alt oug j test 

pothLs of a broad type, he did not bring ihem to a dehn te tes 
Luline Young, in her study of the Molohan sect in ^ngete (63) 
likewise employed several approaches Passwe 
source of most of her documents She also drew “ 
vation, newspaper accounts, and public recor s P 

used delinquency statistics to validate a hypothesis that she formu 

lated from the documentary materials pynressive 

Although much of the research done with 
documents has been rewarding, it is ™ ^ 

gone through the scientific process to the P°‘"' ° developed 

Ls on data different from those with which 'hey -ere d ve^op d 

Although this ,s very expensive and " 

really know the value of this lesearch technique unc 

does this 


REGISTRATION AND CENSUS DATA 

A great body of statistical data about 
collected by government, business, and pnv research, 

frequently, when individual J Although caution must 

drey ignore data available from such sou furnish valuable aids 

be observed in the use of data, they 

to many phases of social psychological rensus data 

bon of such data divides them into registration and census 
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The Nature of Registration and Census Data 

=“ ''«= '™e of the 

reeuiatifin ° accordance with legal or administrative 

rang^ot ev nts and comprise a massive record of social life 
by registration'dafa ‘"ustrative of important activities covered 

1 Vital events 


2 Education 


3 Crime 


Voting 


Births, deaths, marriages, 
divorces, morbidity 
School attendance, grades, 
performance on psychological 
tests 

Crimes known to police, arrests, 
court actions, prison records, 
parole records 
Registration, voting 

rlrmy service 

R-:rra"z;r 

7 rayrolls, production records, 

>0 formal absentee records 

« tzations Membership, office holding 
This list might be “"iraiitee participation 

essential element m an urban almost indefinitely, because an 
system as a basis for knowlerl^*^*^ elaborate record keeping 
variety of events are thus planning and control A wide 

functmning of .he social system ^ P"« °f ^e 

ay have intrinsic value but ,, ^’";‘"f°™ation about the events 
srh T information also r is greatly enhanced b) 

school enrollment frequenU, n "“"I'd J'”'' example, records of 
occupation of father. pS “le nativity of parents 

scores on psychological .e,t““P«''io-« residence for migrants 
There are a few kinds „7°l®”des. etc 
direct psychological measures For?"”''°" records which contain 

an tagnoses may be a sysiemati** psychological test results 

part of the records of schools 
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prisons, mental institutions, certain courts, and personnel depart 
ments of many business firms Usually, however, such psychological 
data are not accumulated in the registration piocess, but even when 
they are not, registration data may still be useful in social psycho 
logical research 

A census is a periodic collection of data about a population, 
usually taken by a house to house canvass In the United States and 
other urban societies, the census has become the source of great 
masses of data about human behavior Although the principal col 
lection agency for this kind of data in this country is the United 
States Census Bureau, valuable data are also widely collected in 
school censuses, in real property inventories and in occasional enu 
merations by other agencies 

The United States Census covers an enormous range both in 
terms of data collected and the types of geographical units for which 
they are tabulated^ For example, the questions in the 1950 decen 
nial census of population and housing cover such items as age sex, 
family composition, labor force status, nativity, amount and source 
of income, education, migration status, type of dwelling, state of 
repair of dwelling, rental, etc For the 1940 census some items of 
this type ^vere tabulated for units varying in size from the United 
States as a whole to individual city blocks of large cities Cross tabu 
lations of many of these variables were published for a variety of 
geographical units The census tract data available for small areas 
of large cities have served as an especially important basis for many 
social science studies {17, 41, 47) Historical comparisons with earlier 
decennial census materials may be made for some variables In a few 
cases these go back to the first census, made in 1790 

In addition to the decennial censuses of population and hous 
ing, significant data are available in the Census of Business and 
the Census of Manufactures Data on institutional populations are 
collected in connection with the population census 

The Census Bureau has recently been collecting and publishing 
certain current types of data on a sample basis At the present time 
these appear under various series of Current Population Reports, 
which include monthly reports on the sizes and composition of the 
labor force There are occasional reports on such subjects as family 

1 For a summary of the scope of the poptilation census see Hauser (28) 
There are occasional guides to available census data (54 58) 
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composition, migration sntus, income, education, characteristics of 
we mg units, etc Most of these are national data, but occasionally 
they are broken down on a regional basis 

noss hi great mass of published data, it is frequently 

rjenro Unpublished tabulated data for the cost 

1 Censt“n™ special tabulations will be made by 

of mo rn ,hf'?“ Tl'e introductory sections 

the scope of tahi l°?Vh * "^“ennial census contain references to 
scope of tabulated but unpublished data 

monographs™re'°ssu^'h”^h'^'^"' and substantive 

pohtfin Districts in tl n Grou»(/j o/ Afff/ro 

'J-stricts m „,e Un„ed Sm,„, by yyi,„en Thompson (55) 

0 / Reginmtwn end Census Data 

different places and^eli?/ clata are collected in widely 

comparison of the inciden«.''°t''h ■'■nc Tins permits the 

spectlic characteristics in d ff '*"* events for persons with 

fcrent temporal conditions' Fo'"' *'""'** P’^'” 

compare birth rates, schncyi ^^^mple, u may be possible to 
of the population voting m elect percentage 

between rural and urban purchases, etc , as 

various phases of the business ctHc 
A nother use of re ' 

ship between nonpsycholovi^i**^*^ study of the relation 

!h° r™ ®°cial psychological rese^^u'***'^ relevant to a research 
the framework provided\v "tty 'hen be pursued within 

ttfonal variables between social and situ 

A classical example of .k 

social variables is a study bv Dur^”^ cegisiration data for relating 
of suicide was related m mem^J^h™ 'he tncidence 

These groupings were then claw “oial groupings 

social cohesiveness The r«„„ =>™rdifg ,o them 

degree of social cohesiveness to m ”'PP°" '“e a theory relating the 
Durkheim himself did not puisne ^ '"whence of suicide Although 
ogist given such background maipn ? further, a social psychol 
® might study this relationship 
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with reference to individuals by examining the psychological cor- 
relates of social cohesiveness as intervening variables. 

A number of sociologists have made studies of deviant behavior 
in which registration and census data have been used to obtain 
general sociological relations and personal documents and case 
records have been used to get at psychological processes relating to 
such a framework (11, 35). Clifford Shaw (43, 44, 45, 46) followed 
this pattern in studying both ecological data relating to juvenile de- 
linquency rates and personal-history documents of delinquents. 

The possibilities in combining current survey data and regis 
tration data have hardly been explored. For example, for a sample 
of precincts it would be possible to combine survey data on social 
participation, reference groups, and political activity with registra- 
tion data on voting, split ballots, registration for voting, campaign 
expenditures, etc. In this way a study could be made of the relation 
between attitudes and social organization of local populations and 
their political behavior. 

Another example of the use of registration data is Klineberg's 
study (31) of selective migration and intelligence. This study de- 
pended on school grades, intelligence-test scores, residence histories, 
and demographic data found in school records. 

Closer to the usual interest of the social psychologist is utiliza- 
tion of registration data to select a sample with specified charac- 
teristics for a study in which additional data will be gathered. 
From the point of view of experimental design, this may have two 
purposes: (I) to select a number of groups which are similar with 
respect to important characteri'siics; sucJi ''equated"' groups may 
then be divided into control and experimental groups to eliminate 
the equated characteristics as variables in the study; (2) to select 
groups which differ on a characteristic which is a variable under 
study in the research. 

The first purpose is illustrated by many studies (27, pp. 57/?.) 
with school children in which they are divided into control and 
experimental groups on the basis of school records. Similarly, in the 
film experiments reported in Studies in Social Psychology in World 
War II (29), the control and experimental groups were chosen by 
matching military units which had roughly similar characteristics 
according to Army records. 
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The second purpose is illustrited b) investigations in the field 
of industrial relations in which plant records of the worker's pro 
ductivjty, demographic characteristics, and work history may lie 
related to current survey or observational data (15, 22) 

Another type of research is illustrated in the study by Cliapin 
and Jahn (14, pp 41 50) in which data from WPA and relief records 
were use in conjunction with morale measurements made by the 
mvesttgators to study the effect of type of rel.ef on the morale of 

htrr'l 1 “ of on wa. 

mterrn f 0" ‘>>0 records of war bond sales as the 

the nr ” ^ various types of studies at least one of 

rnotC? «“■' reg.strat.on data while 

anotte svas measured directly by the invest, galor 

has a ceoera^h ihe case, a field research investigation 

somL 0 ?dlr“ PoW'ra.ions provide a vaLble 

involved tnneinl about the population in the area 

tant basiffor thrcomT^^'' lodicatcd that census data are an impor 
be used to select areas or comm”^ samples Census data may also 
In the study Puh;,, d, * mtmities as the units for research study 
(oO) a group “/ ^ner^ 

matched on census d aromic energy mstalhttons were 

ties removed from $ 11011 ™'^"^' n"” ^"“'’’'^r sample of communi 

members of th p”. ‘"'“"■“'oos Then certain attitudes of the 

Ldre regutrauon 

relationships among social a^T" 

problem Such data cenerall variables relevant to a 

psychological variables H ^ ° mclude direct measures of 
ground variables are freouTmi'^^*^ relationships among back 
ogist and they may facilu ^ “^portant for the social psychol 
example in a study of sociar of other data For 

purchases (49) it was useful t ^ factors affecting home 

data on the tenure status of ° analysis to current census 

Unlike registration da.aT‘wTr"“ 

census data for individuals direct! ^^"erally possible to link 

data about these individuals beca^ o^her social and psychological 
infoimation about indu,f!.,-.i .. Census Bureau must keep 

respondents completely confidential 
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However, through "matching” studies it is sometimes possible to 
combine census and other data as follows (36, pp 14 15) 

The Bureau ol Census uill compile statistics from specific 
census schedules of lists of consumers furnished by business con 
corns A given firm may provide certain information about us 
clients which is transcribed and placed on punch cards along with 
information taken from the census schedules of the same persons 
Cross tabulations are then prepared to show relationships between 
the characteristics known to the census and those known to the 
enterprise No data are given out in terms of individuals but only 
in frequencies or summary fonn 


Historical Series of Registration and Census Data 

For general background purposes as well as for analyses relating 
different time senes of events, historical senes of registration and 
census data may be important The use of time senes in economic 
research is highly developed A considerable number of sociological 
studies have related time senes of social events to economic cycles, 
wars, and other crisis events 

Although lime series involving direct measurements of psycho 
logical variables are rare, there have been comparisons over time 
based on asking the same altitudinal quesiions in several surveys 
(8, pp 220 232) Such series are likely to become more important 
as the number of annual surveys with some continuity in questions 
increases For example, such data arc beginning to accumulate for 
psychological variables affecting consumer decisions in the annual 
Survey of Consumer Finances conducted by the Survey Research 
Center of the University of Michigan for the Federal Reserve 
Board ^ Similarly, a number of survey organizations have been 
collecting comparable attitudinal data about foreign poliq in sue 
cessive surveys It is likely that such time series will grow in number 
Their importance will be greatly enhanced when they cover a suffi 
cienc time period to make it possible to relate them to established 
time senes of census and registration data 

The extent of the histoncal statistical series already available 

2 Report < on ihcvc viir\c)8 nppeir frcipicnth m The Federal Heterve 
nulletm 
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The second purpose is illustrated by investigations in the field 
of industrial relations in winch plant records of the isorker's pro 
ductivity, demographic characteristics, and work history may be 
related to current survey or observational data (15, 22) 

Another type of research is illustrated in the study by Chapin 
and Jahn (14, pp 41 50) in which data from VVPA and relief records 
were used m conjunction with morale measurements made by the 
investigators to study the effect of type of relief on the morale of 
he r«ip.ent Studies of the effect of certatn types of appeals on wai 
™er,o f ‘h' ’■“Ofds of war bond sales as the 

th " '■“'■‘“hie In these various types of studies at least one of 
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However, through “matching” studies it is sometimes possible to 
combine census and other data as follows (3G, pp 14 15) 

The Bureau ol Census lull coinp/Je statistics from specific 
census sdiedules of lists ol consumers furnished by business con 
cerns A given firm nuy proudc certain information about its 
clients which is transcribed nnd placed on punch cards along with 
information taken from the census schedules of the same persons 
Cross tabulations are then prepared to shoiv rel ttionships betiveen 
the characteristics known to the census and those known to the 
enterprise No data are gi\en out in terms of individuals but only 
m frequencies or summary fonn 


Histoiical Series of Registration and Census Data 

For general background purposes as well as for analyses relating 
different time senes of events, historical senes of registration and 
census data may be important The use of time senes in economic 
research is highly developed A considerable number of sociological 
studies have related time senes of social events to economic cycles, 
wars, and other crisis events 

Although time series involving direct measurements of psycho 
logical variables are rare, there have been comparisons over time 
based on asking the same altitudinal questions m several surveys 
(8, pp 220 232) Such senes are likely to become more important 
as the number of annual surveys with some continuity in questions 
increases For example, such data are beginning to accumulate for 
psychological variables affecting consumer decisions in the annual 
Survey of Consumer Finances conducted by the Survey Research 
Center of the University of Michigan for the Federal Reserve 
Board ^ Similarly, a number of survey organizations have been 
collecting comparable attitudinal data about foreign policy in sue 
cessive surveys It is likely that such time senes will grow in number 
Their importance will be greatly enhanced when they cover a suffi 
cient time period to make it possible to relate them to established 
time series of census and registration data 

The extent of the historical statistical series already available 

2 Reports on these surveys ippeir fretjuemh in The Federal Reserve 
tiutletm 
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IS indicated by the fact that approximately 3000 statistical time 
series cohering various periods from 1789 to 1945, have been pub 
Iished in the volume Historical Statistics of the United States 1789 
1945 (57) An appendix in the Statistical Abstract of the United 
States now brings many of these scries up to date 

The following statement from the census volume on historical 
s a istics ( p VI ) serves to emphasize the cautions already ex 
presse m connection with the use of such historical senes 

identification of changes in concept and coserage over 
a pen id ol time is important since such changes may affect sttally 
1 3 span of years Coupled 

in 1 ^ i!^ definitions of terms employed in pub 

h hed historical tables definitions sihich may be in a separate 
publication or may never have been published 


The Use of Published Indices 

bination or othei"m7n published indices based on the com 
Illustrations are the Cw TfT* “nd/or census data 

ness conditions mnnf i \ Index various indices of busi 

fates the FBI* mm ^ indices, juvemle-dehnquency 

rates (birth deatretcr''" ' 

current validity 's*’a'^ne"afnroble’ comparability and 

hkely to be espeaallv 1 ^ historical senes and is 

ludex (60) svhtch rs etm. 2 o" 

a fixed list and quantitv of ^ "Measure of changes in the cost of 
Items was originally basL onT*'""” ®°°* weighting of the 
earners and clerical workers consumption of wage 

were revised on the basis of * items and weights 

income families in 1950 F ^ **'*’^y Expenditures of moderate 
It would be desirable to kee^n comparison over time 

that only the cost of buyine 2 "'eights fixed so 

However from the point of ** goods would vary 

result in an unrealistic index validity this might 

not represent current consumption'' hli, weights did 

•laoits In practice a compromise 
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IS made between the conflicting goals of comparability and cm rent 
validity The items and weights are revised from time to time but 
not each year When the revision is made, both the old and the 
new senes are computed for a while for linkage purposes 

It IS frequently possible for the individual investigator to con 
struct indices for a specific purpose by combining or manipulating 
senes of published data Although sudi indices usually do not meet 
rigid scaling standards, they are frequently useful as rough measures 
of a variable Examples of such indices based on registration and 
census data are those for “plane of living ’ (25) moral integration ’ 
(6), and “segregation (30) Ingenuity on the part of the researcher 
frequently enables him to use such indices as rough measures of 
social variables In many field situations the range of variation is 
so great that even rough indices will serve to differentiate it a 
satisfactory level 

The Ex Post Facto Design^ 

Registration and census data frequently provide data for con 
trasting groups which have already been diffeientially subject to 
some stimulus Studies based on such data have become known as 
ex post facto studies Greenwood (27) defines the ex post facto 
experiment as one in which “ we work backward by controlling 
after the stimulus has already operated, thereby reconstructing what 
might have been an experimental situation This is to say that the 
stimulus is not controlled by the investigator The nature of the 
experimental manipulations the investigator can make are strictly 
limited 

The Chapin and Jahn study (14) of the relation of type of 
relief to worker s morale cited earlier in this chapter, is an example 
of an ex post facto study based on registration data In this case the 
stimulus (type of relief) was not controlled by the investigator A 
group of persons receiving work relief through the WPA was matched 
with a group receiving direct relief but eligible for WPA work 
relief The characteristics used for matching were age sex race, 
nativity, amount of education, usual occupation sue of family, and 
length of lime on relief Morale measurements were made on both 

3 Students interested in an intensive stud) of the ex post facto design should 
rnnstiU Greenwoods (27) carctui 3nal>ses 
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groups The group receiving work relief scoiecl significantly higher 
on morale tests than the group on direct relief 

An example of ex post facto »-esearch based on census schedule 
data IS the study by Freedman and Hawley (23) of the relation 
between unemplojment and migration In tins case the migrants 
in two Michigan cities were matched on a number of reletanl 
«nsus characteristtcs with nonmigrints at their place of origin 
e two groups stere then compared on unemployment lates prioi 
warirT?" 1,^*1' P^nmigration iintmploymeni rate lor llie migrants 
"“""'W-'nlt' The dillcrence was 
ss an a expected on the ilieoiv that uneinployiiiciit 
was an important cause of migration for con.paiable groups 

been don^J ^“SS«>'ve ex post facto studies base 

number of firin'"" '^i''*' control variables 1 he largest 

excellent study^Xd “mad^T' 

investigate an 1 ■bgenious use of registration data to 

fhe b^STuleCri " ^ 

sonal and social contrils Thn s, IT''”," P" 

including psychiatric d,, materials 

ers. for lUO luvemlp rf i records of social work 

court records On the ha probationers m the official juvenile 

the folic, oinch!. “ data Re.ss reached 

tant relationships whicrmav be'' ’’lustralive of types of impor 

designs ^ ^ mvesiigaied by various ex post facto 

often than non ***'’' reciduists are less 

social milieu wlJch of socn] groups and Ine in a 

uiques m omHi 'O’^ctenied by norms and effective tech 

(2) that del.nquemT behavior contra delinquency 

-trol Of SO^I ' - -bmit to the 

than do non recidnists ' enforce such conformity behavior 
less often persons With* delinquent recidivists are 

roles and appropriate and^fll^ delinquent socnl 

the individuals to guide * rational controls which permit 

social group expectations (sTp^oV^”"'* with non delinquent 

The most important limiiaim 

kind is that the members of «k " facto studies of this 

e control and experimental groups 
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are ‘ self selected” rather tinn randomly assigned This means that 
the differences found between the control and experimental group 
may be due to factors connected with characteristics of these groups 
other than the experimental stimulus or the characteristics matched 
For example, in the Chapin and Jahn study (14) such factors as 
persistence or political activity may conceivably explain both the 
stimulus variable (type of relief) and the effect variable (morale) 
Despite the matching process, it is possible that the tivo groups 
differed in morale even before they iiere subjected to different relief 
programs Similar uncontrolled selective variables may account for 
the results of the Freedman Haivley study (23) Even with this 
limitation, the ex post facto design may be useful in field studies, 
especially if replication under various conditions is possible It is 
important, however, to understand the limited level of generaliza 
tion which IS usually possible with this type of design 

Problems m the Use of Registration Data 

The fundamental limitations of registration data for social 
psychological research arise from the fact that they are not generaJI) 
collected for the specific purposes of such research The definitions 
and tabulations used in calculating and processing the data may 
differ from those which the researcher would use in collecting data 
for his own purposes For example, the detail iwih which occupa 
tional data are recorded in school files may not be ideal for the 
investigator who wishes to use occupation as an index of social class 
Most of the limitations of registration data come from the basic 
fact that the investigator cannot impose his own standards of valid 
ity and reliability on the data 

The completeness of coverage of registration data varies from 
time to time in accordance with the efficiency of the data collection, 
the nature of the data and the incentives which the population 
has to record the event involved For example, it has frequently been 
observed that a recorded rising birth rate in a country undergoing 
modern development is not necessarily proof that the birth rate is 
actually rising It is more likely to indicate that the efficiency of 
birth registration has improved along with other statistical services 

Similarly, variations in the reported incidence of some regis 
tered event may reflect variations m the vigor of data collection 
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rather than variations in the e\ent Itself kuczynski (32 p 8) reports 
that extreme variations in vital rites of certain countries resulted 
from periodic campaigns to register events not recorded in previous 
periods The extreme irregularities in these rates were largely sta 
tistical artifacts 

Because registration data are collected as an incident of admin 
istrative processes they may sulFer from their particular context 
ncome tax returns have obvious limitations as accurate measures 
countries because of the motivations for tax 
oiipni^a ° pointed out that the chance that a delin 

and eth^ ^ recorded is related to the social status 

Similarly Sutherlands 
muted hv ^ crime indicates that certain crimes com 

sider whetheAhe^j!''^ '’alidity ot data the investigator should con 
and whether the "““"o 8‘''« the information required 

■nfomtatton accurately LunSrergmTif.t"'’'*"”" 

times the cause of dL^t. ? ' . P reports that at various 

as died suddenly n u ^catli certificates has been recorded 

vita. s.aA»: VAefe'A™' 

by unqualified lay persons Ev^n^ 

by the state of knowipd qualified personnel are limited 

certain causes ate bte ^eVto'b' 

skill in diagnoses of ft, ^ partly attributable to an increased 

In P-fession 

the recorder is not mot necessary information but 

For example where ethnic^h Properly instructed to elicit it 
the record of some event th S^ound is obtained as incidental to 
relevance for his agency o ? ™uy feel that the data have no 

and record the data Fortun^tT^^ improperly trained to obtain 
agencies are training record ^ ^ increasing number of cases 

those given to the interviewp^f instructions parallel to 

In using time senes of ^ schedule 

suie that regulations coverintr tJ***°" important to be 

viously comparisons of the n h have not changed Ob 

period of time is no indication™ income tax payers over a 

income subject to taxation has ch *”*'**'^^ levels if the minimum 



Documents, Records, Census Materials, and Indices 


321 


A special problem of this kind may be changes in the practice 
of assigning the event to the place where it occurs or to the legal 
residence of the person to whom it occurs. The registration of births 
and deaths is a case in point. At the present time these data are 
tabulated both as to place of occurrence and place of legal residence. 
That such allocation of the events is an important issue may be seen 
from the fact that in 1940 21 percent of all registered births were 
coded “nonresident." This percentage varied from 8 percent for 
rural places to 40 percent for births in cities of 2500 to 10,000 in 
population (61, p. 17). 

In some cases the reporting agency may have made studies 
to test the reliability or validity of its data. In other cases, the investi- 
gator himself should make such tests whenever possible by checking 
for internal consistency in the data or by comparison with other 
series. One indication of the coverage of a registration series is the 
1950 check comparison of birth registration with infants, reported 
in the 1950 U. S. Census (42). For the United States as a whole, 
birth reporting was estimated to be 97.8 percent complete. Estimates 
for individual states ranged from 88.1 percent for Arkansas to 100.0 
percent for Connecticut. It was less than 95 percent in only seven 
states. The 1950 figure of 97.8 percent for the United States as a 
whole represents marked improvement from the 1940 figure of 
92.5 percent. 

The extent to which registration data are subject to the limita- 
tions we have presented varies widely with the administrative con- 
text in ivhich they are collected. In some cases, the collection is 
mtelligently and specifically guided by the research purposes of 
the agency or for general research purposes. However, it is obviously 
important that the researcher who uses registration data should 
know something of the special definitions and contexts which may 
affect the nature of the data collected. The fact that data are pub- 
lished under official auspices does not guarantee their quality. 


Problems in the Use of Census Dala 

To a certain extent, the census, as a source of research data, is 
subject to the same limitations as registration. Most important, the 
definitions used in data collection may not always be completely 
appropriate for the purposes of a specific piece of research. It is 
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also true that over a period of time changes are made in the classifica 
tion system and these may produce errors in interpretation if they are 
not clearly understood For example, between the 1930 and 1940 
census the basis for identifying the working population was changed 
In 1930 and earlier years, tlie working population was identified 
ivith gainful workers— persons having a gainful occupation regard 
less of ishether they were actually working or seeking work at the 
time of the census, since 1940 the ' labor force concept has been 
use to classify workers on the basis of their activity in a specified 
period at or immediately prereding the census 

The importance of understanding the working definitions by 
hv illustrated in a recent sample survey 

y e Census Bureau (56) In this survey 844.000 persons classified 
^'^d nonfarm residences had not 
rnifrnme'^ physical location Their classification as 

they were re^!?^ ^ change m the use of the land on which 

ment based jmilarly, estimates of the amount of unemploy 

^f wharrnn,?.'?^ ' to the definmon 

family workers ^ ^!i employment, the status of unpaid 

are unemDlQv^>H demarcation between those who 

document thl ^ *" force These illustrations 

- .. 

high lev^el ^ ^ *^e«suses are of a relatively 

Bureau includes an outstandiV'^*'’' personnel of the Census 
scientists ^Vithin thf. i ^ KJ’oup of statisticians and social 

t>.e Census'^tLu rr'.T 

odological aspects o[ survey work Wh'" T"’' 
exist, they are usually pu,n,,H ^ 

has been an increasiUE eHort i„ ' publications There 

the dua into the operations of j]"^P°’^te ‘:hecks on the quality of 
1 oi me Census Bureau (34) 

SUMMARY 

This chapter has dealt with the 

of data collecttxi by nersons „ti, c' *" ''''^'^^eeh of certain types 
Such data are asailable in creat oL^w -nvestigator himself 


great quantity and for a wide range of 
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important problems as a result of the elaborate record keeping and 
documentation m our society The principal limitation of such data 
IS that the operational definitions of the data and the possibilities 
of experimental manipulation are outside the experimenter’s control 
Although this restricts their usefulness, such data remain very impor 
tant They provide unique access to historical social situations and 
to some current social situations which are othenvise difficult or 
expensive to observe Moreover, these are data in a “natural social 
setting 
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The Collection of Data 
by Interviewing 

Chmles F. Cannell and Robert L Kahn 


In almost every field o£ human thought .t .s possible m obsem 
indications of the laborious ascent from supersti i 
to scientific fact Such observations reveal that imp ^ ^ 

systematic collection of data is a major -characteristic of science d 
velopment In the long established Phy--> 

and techniques o£ data collection are '> and auanti 

sciences the development of techniques for tneasurem n and quan.i 
lication has recently become a focus of effort an a 

To some extent the needs of the measurement 

met through techniques of °hser”tmn and^pjs.^ demanding data 
To an increasing degree, however, soc experience 

which must be reported by ‘"‘’"'“’" Jed behaiior, are avail 

Attitudes, perceptions, expectations, -y P anthropologist 

nble to the economist, sociologist, psycho g . 

only through such direct communication .i,^avs • communi- 

In a sense, of course, social saentis rommunications 

rated" with people and derived insights highly subjective 

The problem for social science is to method for the collec 

process of "getting insights" into a sjste principles and 

tion of social data This chapter discusses some 
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techniques by which the process of interviewing can be made to 
approach the criteria for suentific measurement 


Criteria for Scientific Data Collection 

The adequacy o£ a technique for collecting data is ordinarily 
ju ged in terms of criteria of reliability and validity, concepts which 
are discussed at length in Chapter 6 Reliability requires that re 
peated measurements yield results which are identical or fall within 
narrow and predictable limits of variability The criterion of valid 
ity demands that the measurement be meaningfully related to the 
esearc o jectives that is, that it measure what it purports to 
measure ^ ^ 

men! ^^^sc ctitena apply not only to the data collection instru 
the morn * technique and procedure spectfied for using 

not onl „ uTb “<* *o°al data depend 

but also itnon^ questionnaire or interview schedule 

techniQue tff mii ' "’tinner ot administering the instrument, the 
for nordmir m ""''"'‘"S ’r*’® techniques discussed in this chapter 
interviews are eonstructing questionnaires, and conducting 

Tn 7 ^““’"her tn approx.mat.ng the 

twin goals of rehab, l.ty and vahd.ty m lus data coflectton 

Potent, alu„s of the Intennew 

lecttng data rcq'uire'd m T T'll' ’"'«’''‘ew as a device for col 
principles which eov hypotheses in social research The 

the naming of mte^ieirers'*r'^'i?"™"^' '•"’‘gn, interviewing, and 

m which information is desired r''”""''"’ ""“t situations 

must interview his client ^ respondent Thus, the lawyer 

physician must base hi^ represent or defend him, the 

'veil as the examination ‘he medical interview as 

social worker— all depend to^ the personnel officer, the 

'lewcrs as well as upon ^heir skills as inter 

demand of them skills which tlieir professions 

The fact that the 

that It ts the best device for collel^r'* "'"inly does not imply 

stances One of the choices winch til? 

the social scientist must make 
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involves the various methods of acquiring data One of the impor 
tant criteria is the relative accessibility of the required data to dif 
ferent means of collection The sources and disposition of fami y 
income provide an example of data which are at present virtua y 
inaccessible from sources other than personal interview 

Suppose, for example, that our research objective is to test 
some hypotheses about the relationship between the source an 
amount of family income and the pattern ot saving and spen mg 
This objective requires that we assemble data on mcome an ex 
penditures for individual families Although gross expen 
different commodities might be estimated from data provi e y 
ufacturers or trade establishments, and the volume of savings migh 
be determined from banks, the pattern of income and expenditure 
of family units cannot be reconstructed from external sources uc 
information is uniquely available through interviews with a sample 


of family units _ j 

On the other hand, there are many data relating to income and 
expenditure which can be obtained with accuracy and econo y y 
means other than the interview survey We might, for examp e, 
to test the hypothesis that sales of government bonds P ^ 

roll deduction plans tend to increase after a plant vvi ^ ^ 

mem If this is our research objective, it is like y t at Qg-^rt 

records, perhaps in combination with those of t e re ^ ^ 
ment, can best meet our need for data The alternative o i 
mg relatively large numbers of people in order to i en i y 
bond buyers and then questioning this group a ou * . 

purchase and recent fluctuations m income is cost y an co p 


^TnodieTkmd of data which has been successful!) 
n^eans of mteiview and personally administered 
lias to do with the attitudes, perceptions, and e avior ° P ^ 

"ork situations For example, we might study t e VP® ,„,rmsic 
"other’s motivation to produce will be relate to 
snislaction which he derives from his job Or- we may 

>hai worker productivity depends upon the indis I i ^thc extent 

o' the consequences o£ high or low prodiicivit). and t^c ex.c^^ 
to which these consequences represent personal goa s _,i„ uiml 
O' these hypotheses requires data which arc msi e .jicr ap- 

and which he alone is capable of communicating A ) 
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proach to assessing the indi\ idual s satisfaction with his job ivould 
almost certainly involve a risky process of deduction and inference 
Even when the research objectives call for information which 
IS bejond the individuals power to provide directly, the interview 
IS often an effective means of obtaining the desired data The studies 
of prejudice and ethnocentrism by Adorno et al provide an example 
of such research (1) The research design called for the rating of 
individuals along a number of dimensions, including anti Semitism 
and other ethnocentric characteristics, politico economic conser 
vatism, and several aspects of personality organization^ Bias and 
lack of training make it impossible for an individual to provide 
directly and with validity such intimate information about himself, 
even d he is motivated to the utmost frankness But only he can 
provi e the data about his attitudes toward his parents, colleagues, 
an members of minority groups, from which some of his deeper 
lying characteristics can be inferred 

In short, if the focal data for a research project are the attitudes 
fn. individuals, the most direct and often the most 

^5k the individuals themselves As Jahoda, 
1 suggest, observational methods are of pri 

in studying behavior which takes place 

tionai situition, in response to known stimuli Observa 

attiturl ' for measurement of 

or to Perceptions and are obviously unable to probe the past 

tcria oi d.r'I!','"' s mtemions for the future The cri 

beliefs economy, and the ability to collect data about 

the rant»p of experiences, and future intentions have widened 


Lmuations cf ,hr Imcrv.m 

of titc inthltdral'lrih^'u"’ “5 ‘"'"'‘ow ts 'he involveme.it 

likelihood of bias Even .f ,1 "'O consequent 

Sion of certain fart« h ^ ^ assume the individual to be in posses 
certain facts, he may withhold or distort them because to 

* In this series of stuiliM 

for iJie .Ickclopmcnt of lunoclicse^^rf . c^P'oraiory purpose 

obtainetJ U\ ivrittcn quotioniiai^ ^ i««rumenis ind for salufation of data 



Collection of Data by Interviewing 


331 


communicaie them is threatening or in some manner destructive 
to his ego Thus, extremely deviant opinions and behavior, as well 
as highly personal data, have long been suspect when obtained by 
personal interviews Nevertheless, much experience in recent years 
indicates that such limitations on interview subject matter are 
not to be ngidl) assumed 

Another limitation on the scope of the interview is the inability 
ol the respondent to provide certain types of information For exam 
pie, the hypothesis that paranoid tendencies are related to an ina 
ity to \\ork with groups demands some measurement o tie re 
spondent’s personality structure Although he may be comp ete y 
unqualified to make a direct judgment of such characteristics 
himself, he is uniquely qualified to pro\ide some persom m orma 
tion from which an expert might make a diagnosis Thus, t le ina 
»iy of the respondent to provide a certain datum may mean t a 
different means of data collection is advisable or it may mean 
die interview must be so constructed that the responc ent prov 
raw data which are relatively available and nont reatening 

so that experts may then interpret his responses m order 
provide the information specified by the research objectives 

Memory bias is another factor which renders the respondent 
unable to provide accurate information Often, t e on ^ ^ , 

around the problem of recall is to have the foresig t an 
to carry out a research design over a period of yt . 

priate measurements at tlie time intervals indicate y 
objectiv es 

In summary, the interview and questionnaire appear P 
ful instruments for social research, and the range o t eir u 
ts steadily widening Individuals' past experiences an 
havior are virtiiallv unobtainable by other means I 

attitudes, 


^ widening Individuals past expenc p^rrentions 

:e virtually unobtainable by other ^rvation aie 

and opinions which cannot be inferred y o ^ „ 

^cessible through interviews The major problems m mte 
from the inability or unwillingness of the respon 
municate These problems, as we have seen, can j 

"holly o- w The skills and technique ot 


die interview 


te These problems, as we and technique of 

or in part by various means The skins ai ^ ^ , 

irviewer, the ingenuity of the dan collecting in 
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ihe knowledge of the analyst can compensate to some degree foi 
the biases, memory failures and inexpertness of the respondent 


THE PSYCHOLOGICAL BASIS OF THE INTERVIE\V 

In discussing the process of collecting social data, we have 
implied that the questionnaire is a measuring instrument or device 
used by the social scientist in much the same way that specialized 
instruments of measurement are utilized in other fields The inter 
vieuer is a technician who manipulates the instrument, takes the 
appropriate readings, and records the results In this sense the inter 
viewer s function parallels that of scientific technicians in other fields 
First, he must be provided with a questionnaire which is adequate 
to the research objectives Secondly, he must ask the questions and 
record the responses in a standard way 

Considering the interviewer as a scientific technician and the 
interviewing process as a scientific technique implies that we are 
1 e, through the application of a specific instrument in a specific 
manner to achieve identical results in given situations There is 
further implication that we are able to specify explicitly each step 
technician must follow in using the instrument 
** sufficient similarity between the scientific technician 
u ® to make the analogy attractive however, it does 

no o completely If the interviewer were only to ask a specific 
^ way he would not succeed in obtaining 

of fnni respondents which reflected the same degree 

li e and so on In short, 

- - - 

social mvrhAiof»i,->T f hand, must take account of such 
n s uSLs « th«e, and >ho measure of 

e "len m ssh irh' '"Bely dependent upon the 

to «h.ch he IS insightful and successful m recognizing and 
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dealing \vith the social-psychological phenomena of the interviewing 
process. 

Contemporary social science does not provide the interviewer 
with adequate methods for dealing with all the variaD es at wor 
in the interview. To some extent this might be t oug ^ ^ 

symptom oE the youthful inadequacy of social science in genera an 
social psychology in particular. To a considerable extent, ouever, 
it is also a function of the unusual complexity of the subject matter 
on which the interviewer, as a scientific technician, is exercising 

his techniques. t i, i. 

Much of the available literature consists of rules of thumb p 
tented as lists of ■'do's" and "don’tV for the interviewer and for 
the questionnaire framer. These do’s and don’t’s are essentia y 
systematic compilations of interviewing experience derive ro 
variety of situations over a considerable period of time. " ? 

regard them as the "folklore" of interviewing, based on exper . 
and thus as having a good deal of pragmatic uti iiy. J 
represent practices which have achieved a degree o 
variety of situations. Nevertheless, they have the isa jnte^iew 
being somewhat unsystematic in their approach o 
process. There are tew scientific studies testing these com • 
injunctions, and one must accept or reject, wit out pr ’ 
those interviewing principles whicli other peop e a | 
work. A final disadvantage of the common-sense ru es o 
'ng is that at best they represent a superficial statemen 
stitutes successful interviewing procedures. They o 
nnderstand the interpersonal relations between m ^ 

respondent. They do not tell us why a specific practice ^ 
successful or unsuccessful interview, or within what ra g P 
practice is desirable or undesirable. , inter- 

Until we have a theoretical basis for jnter- 

process, and until we have tested empinca y so 
Viewing folklore which we frequently take for gra 
unlikely to advance in a basic way our knowledge jo 

■nterviewing procedures. In other words. research 

spect that we possess a powerful limitations, 

^•a. hut we do not yet know its full potenti comprelien- 

Unfortunately, social science has not yet provi 
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tion, although insumcient for an adequate interview, at least enables 
the interviewer to describe the piojcct and permits him to ta e 
the first steps toward building sufficient motivation to permit t e 
interview to move fonvard The initial reactions oi respondent vary 
considerably among different segments of the population an t ese 
variations are reflected in the refusal rates recorded by various sur 

vey agencies For example, the higher rate of initial icfusals m ur an 

areas illustrates a difference between urban and rural popu ations 
with respect to attitudes toward the casual caller and the appropriate 


behavior toward him 

For some parts of the population, the customary ^ 

reacting to authority figures may do much to determine t e mi i 
reaction to an interviewer Thus, the interviewer may gain acces 
to a respondent because he is perceived as an indivi ua o 
a representative of an agency possessing authority an ° 

respect from the respondent For example a respon ent 
favorably to the interviewer because he represents a we 
research organization, a university, or a governmental 

The interviewer necessarily accepts these motives as a 
beginning to communicate with the respondent 
mediately begins to define the situation in a manner w .Upnch 
the interview to certain goals the respondent is 
2nd accordingly, gives the interview a positive vae 


respondent « 

tn some studies, the relating of the iniert lew to the 
goals may be started in advance of the interview itse y 
Inters, radio, or newspaper announcements The “ 

duction and statemenVof the purpose ot the ‘as 

tended to make the interview appear compati e 
2 means of achieving, some respondent goal beeins 

After intnal acceptance by the respondent, 'he .nter lew bjns 
'vitli questions designed to develop active ° to as 

*0 respondent These are the kinds of items of n,otivrte 

rapport builders ’ The purpose ot such '1“^* interesting 
0 respondent by assuring him that the interviev , tviiich he 

■■'hat IS that its content is related to interests “ S j^sttons 

"'ready has An additional purpose ot these rrr'r^du^^ 

" to relieve anxieties which the respondent may _ j This iv 

>0 his own ability to play the respondent role e 
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dotl'i educating him in, or clarifying for him, the type of response 
CHjicctcd, thus demonstrating to him his ability to handle the pre- 
scribed role. If the initial phase of the interview is successful, the 
Inicrvicw has reached the point where one of two major types of 
motivation may be tapped, thereby ensuring continued cooperation 
by the respondent. 

Recent work in communications within small groups has 
resulted in findings which appear relevant to our understanding 
of communication between an interviewer and a respondent. These 
findings can be summarized as follows. One of the motives for com- 
municating is the desire to influence, m some manner, the person 
to whom the communication is addressed. That is, a person will 
communicate in a given situation if he believes that such com- 
munication will bring about a change or effect an action which 
he con, rfers desirable (9). In the interview situation, this means 
e I . er ^ ^ ® interviewer will be perceived as a person who can 

^ u t^imctly or that the interview will be seen as a 
rIiniV.1^ accomplishing a desired change. A 

nerreii Jk' ° a social Worker, or physician is frequently 
Ksnonden^ '''«>« as a direct agent of change. The 

Ins fmanns/rf'ir ^ f 'ci™municating his symptoms or 

d e dv A ' , P™''«ional person svill beLfit him 

viewe^and ' 1 “ ^='atmnship betsveen inter- 

market rcsearci ^ respondent is provided by the typical 

«prcs cToretr'?’ that by his 

acratlsdc o'! a producX^^ f 

net in terms of his nw • i "^*^'**y *tclping to improve the prod- 
"’orld U™ a t eaaranle 

on gosernment programs.lor a'rra’L'T'’' o°'’‘'“o‘c>l 

P®^dcni to Drefarp Iii« rp^nnn^ps 
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lepreseniative of an agenqf which is able to bring about change 
For example, facts about personal income, often considered difficult 
to obtain in interviews, are more readily available to the researcher 
if the respondent believes that the information he proffers wil e p 
responsible people in the government develop policies which will 
contribute to the public welfare and to the well being of t e re 
spondent 

A second major type of motivation depends more directly upon 
the personal relationship between the interviewer and t e re 
spondent It can be defined as follorvs An individual is motivated 
to communicate with another when he receives gratification rom 
the communication process and the personal relations up u 
motivation sometimes occurs because the interview offers tie 
spondent an opportunity to talk about topics in w ic 
interested but which usually do not obtain adequate expression 
This does not imply that the respondent in a researc m 
ordinarily obtains cathartic release (although this ^ P 

at times) It does mean, however, that the respondent obtains satis 
faction from talking with a receptive, understanding m 
about something in which he is interested and ini'/ ic 
volved The reader will recognize this as one of the basi 
patients in the psychotherapeutic interview 
Interviewers are often surprised to encounter t is . v 
m a research interview, m which the possibilities (or ^ 

of a therapeutic type of relationship appear remote ^ j 
shows, however, that if the research interview is ^ P 

^his motivation is often present The relationship m ^ 

msembles the counseling relationship Counselors ^ 

have found that freedom of communication (even t e 
tmn of deep personality conflicts) is possible under t e p 

Rogers^, for instance, identifies four qualities h jms 
characteristic of the productive f interview He 

of these four are relevant to the resea n^nsiveness ’ 
characterizes the qualities as, first, “a warmth and resp 

a the part of the counselor which 'expresses itsel ^ 

'crest in the client and an acceptance of him as P ^55,00 
otTI described as “permissiveness by the 

feeling By the counselor s acceptance of his 

lack of any moralistic or judgmental attitude, by the 
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done by educating him in, or clanfying for him, the type of response 
expected, thus demonstrating to him his ability to handle the pre 
scribed role If the initial phase of the interview is successful, the 
interview has reached the point where one of two major types of 
motivation may be tapped thereby ensuring continued cooperation 
by the respondent 

Recent work m communications within small groups has 
resulted in findings which appear relevant to our understanding 
of communication beti^een an interviewer and a respondent These 
findings can be summariz'*d as follows One of the motives for com 
municating is the desire to influence, in some manner, the person 
to whom the communication is addressed That is, a person will 
communicate in a given situation if he believes that such com 


munication will bring about a change or effect an action which 
desirable (9) In the interview situation, this means 
either that the interviewer will be perceived as a person who can 
ring a out change directly or that the interview will be seen as a 
potential vehicle (or indirectly accomplishing a desired change A 
mica psychologist, a social worker, or physician is frequently 
perceived by the patient or client as a direct agent of change The 
communicating his symptoms or 
professional person will benefit him 
example of a less direct relationship between inter 
market reie * t? ^ respondent is provided by the typical 

expressed nre^ ®“rvey, m which the respondent believes that by his 
acteristic of ■» ^ specific kind of packaging or other char 

net in terms o^^h ** indirectly helping to improve the prod 

World War IT f** wishes and needs It was common during 

onTcv^nmern’pXtrt 

^vlth "You tell til ^ 1 ^ ^^^P’^^dent to preface his responses 

This tvp of Washington that I said . " 

relat.on5h.ps are apparent" The “"‘S' 

the content of the intervmw , (>> The perception of 

The respondent w.ll to a change which he des.res 

project to be related to h.s 

demonstrate th.s relat.onsh.p ^ tu'to T 

.n’erviencrr ; rrsor,rr"T‘^ "" 

P no can bring about change or as the 
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lepresentative of an agency which is able to bring about change 
For example, facts about persona! income, often considered difficult 
to obtain in interviews, are more readily available to the researcher 
if the respondent believes that the information he proffers will e p 
responsible people in the government develop policies ivhic wi 
contribute to the public welfare and to the wellbeing of the re 
spondent 

A second major type of motivation depends more direct y upon 
the personal relationship between the interviewer and the re 
spondent It can be defined as follows An individual is motivated 
to communicate with another when he receives gratification rom 
the communication process and the personal relations up 
motivation sometimes occurs because the inteiview o ers tie 
spondent an opportunity to talk about topics m w ic e is 
interested but which usually do not obtain ^‘>'^‘1“^“ 

This does not imply that the respondent in a researc in “ 
ordinarily obtains cathartic release (although this may e p 
« times) It does mean, however, that the respondent ^ 

faction from talking with a receptive, understanding in 
about something m which he is interested and in which 
Volved The reader will recognize this as one of the basic 
of patients in the psychotherapeutic interview —novation 

Interviewers are often surprised to encounter i , , , 

m a research interview, in which the possibilities (or ^ 

of a therapeutic type of relationship appear remote P j,] 
*ows, however, that if the research interview is conduct P P^^ 

*1S motivation is often present The relationship in 
msembles the counseling relationship Counselors “ 

•'ave found that freedom of communication (even the 
h°n of deep personality conflicts) is possible under t e p p 

Rogers! for instance, identifies four qualities ’''■’.ch he claims 
“e characteristic of the productive counseling He 

^hree of these four are relevant to the research 
Aaracterizes the qualities as, first, a warmth and re p ^ 

" 'he part of the counselor which expresses Use . 

"‘crest ,n the client and an acceptance of him as , p 
™nd quality IS described as “permissiveness in ''"E" the 

"Heeling b' the counselor s acceptance smtements, by 

“"’P'ete lack of any moralistic or judgmental attitude, by 
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done by educating him in, or clarifying for him. the type of response 
expected, thus demonstrating to him his ability to handle the pre 
scribed role If the initial phase of the interview is successful, the 
interview has reached the point where one of two major types of 
motivation may be tapped, thereby ensuring continued cooperation 
by the respondent 

Recent work m communications within small groups has 
resulted in findings which appear relevant to our understanding 
of communication between an interviewer and a respondent These 
findings can be summarized as follows One of the motives for com 
municating is the desire to influence, in some manner, the person 
to whom the communication is addressed That is, a person will 
communicate in a given situation if he believes that such com 
mumcation will bring about a change or effect an action which 
he considers desirable (9) In the interview situation, this means 
either that the interviewer will be perceived as a person who can 
bring about change directly or that the interview will be seen as a 
potential vehicle for indirectly accomplishing a desired change A 
clinical psychologist, a social worker, or physician is frequently 
perceived by the patient or client as a direct agent of change The 
respondent is likely to feel that aimmunicating his symptoms or 
his financial difficulties to the professional person will benefit him 
directly An example of a less direct relationship between inter 
viewer and the goal of a respondent is provided by the typical 
market research survey, in which the respondent believes that by his 
expressed preference for a specific kind of packaging or other char 
acteristic of a product he is indirectly helping to improve the prod 
uct in terms of his own wishes and needs It was common during 
World War 11, for example when surveys were being conducted 
on government programs, for a respondent to preface his responses 
with You tell those people in Washington that I said 

This type of motivation can be developed only if the following 
relationships are apparent to the respondent (1) The perception of 
the content of the interview as relevant to a change which he desires 
The respondent will not spontaneously perceive every research 
project to be related to his goals and interests The researcher must 
demonstrate this relationship or suffer the results of reduced 
respondent motivation to communicate (2) The perception of the 
interviewer as a person who can bring about change or as the 
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representative of an agency which is able to bring about change. 
For example, facts about personal income, often considered difficult 
to obtain in interviews, are more readily available to the researcher 
if the respondent believes that the information he proffers will help 
responsible people in the government develop policies which will 
contribute to the public welfare and to the well-being of the re- 
spondent. 

A second major type of motivation depends more directly upon 
the personal relationship between the interviewer and the re- 
spondent. It can be defined as follows: An individual is motivated 
to communicate with another when he receives gratification from 
the communication process and the personal relationship. Such 
motivation sometimes occurs because the interview offers the re- 
spondent an opportunity to talk about topics in which he is 
interested but which usually do not obtain adequate expression. 
This does not imply that the respondent in a research interview 
ordinarily obtains cathartic release (although this may be present 
at times). It does mean, however, that the respondent obtains satis- 
faction from talking with a receptive, understanding interviewer 
about something in which he is interested and in which he is in- 
volved. The reader will recognize this as one of the basic motives 
of patients in the psychotherapeutic interview. 

Interviewers are often surprised to encounter this motivation 
in a research interview, in which the possibilities (or desirabilities) 
of a therapeutic type of relationship appear remote. Experience 
shows, however, that if the research interview is conducted properly, 
this motivation is often present. The relationship in many ways 
resembles the counseling relationship. Counselors and therapists 
have found that freedom of communication (even the communica- 
tion of deep personality conflicts) is possible under the proper condi- 
tions. Rogers, for instance, identifies four qualities which be claims 
are characteristic of the productive counseling atmosphere ( 18 ). 
Three of these four are relevant to the research interview. He 
characterizes the qualities as, first, “a warmth and responsiveness" 
on the part of the counselor which “expresses itself in a genuine 
interest in the client and an acceptance of him as a person." The 
second quality is described as "permissiveness in regard to expression 
of feeling. By the counselor’s acceptance of his statements, by the 
complete lack of any moralistic or judgmental attitude, by the under- 
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standing attitude which pei\ades the counseling interview the client 
comes to recognize that all feelings and attitudes may be expressed 
No attitude is too aggressive no feeling too guilty or shameful to 
bring into the relationship A third characteristic of the productive 
counseling relationship is freedom from any type of pressure or 
coercion The skillful counselor refrains from intruding his own 
wishes his own reactions or biases into the therapeutic situation 
Applying these criteria to the research interview ue may con 
elude that optimum communication takes place if the respondent 
perceives the interviewer as one who is likely to understand and 
accept his basic situation The interviewer is thereby perceived 
as being within range —that is he is seen as a person who will 
accept the respondents statements and experience This does not 
mean that the respondent must see the interviewer as similar to 
himself but he should view the interviewer as capable of under 
standing or of being completely tolerant of his point of view This 
perception will depend far more on the interviewers iititude and 
the sort of relationship which the interviewer establishes with the 
respondent than on such external factors as the interviewer s dress 
or appearance even though these externals may provide some initial 
cues for the respondent 

Unfortunately a number of factors may inhibit communication 
or distort the content of the information given by the respondent 
For example the respondent may not accept the goal which the 
interviewer describes as the purpose of the survey 1 hus the goal of 
providing the federal government with information on income 
distribution may not appear worth while to some respondents when 
they are questioned about their own Bnances and economic situa 
tion Even more frequently the respondent may possess goals which 
are in conflict with the purpose of the interview In an industrial 
plant for example a worker may be wholly sympathetic to the 
notion that employee opinions should be solicited and he may be 
hopeful that frank expressions may result in improving certain situa 
tions However he may also think that his expression of critical 
opinions may be dangerous and may make him liable to retaliation 
or discrimination He may even be concerned over loss of work or 
loss of promotion Such an attitude does not result in a refusal of 
an interview but it does limit the content areas about which the 
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worker will talk freely. He might discuss physical Avorking conditions, 
wage policy, and the like quite frankly but be extremely guarded 
and reticent when discussing any aspects of his job which might be 
interpreted as criticism of his immediate superiors. 

Just as a respondent may refuse to communicate or may distort 
his communication because he rejects the research goals of the 
interview process, he may also refuse or restrict communication 
because the personal relationship with the interviewer is such that 
real communication and understanding is impossible. This kind of 
respondent reaction may occur as a result of a stereotyped judgment 
which he makes of the interviewer. Thus, if the respondent per- 
ceives a gap of education or an economic difference between him- 
self and the interviewer, he may decide that the interviewer is 
incapable of understanding the respondent’s family circumstances 
or of empathizing in any ivay with his predicament. This problem 
in communication is very likely to arise when the respondents have 
some perception of themselves as deviant in the content area f>f the 
interview. For example, a respondent holding extremely radical 
political views might perceive the intervietver as necessarily tllffering 
so greatly from himself that no tolerance for his point of view is 
possible. In such instances, the respondent has in effect concluded 
that the interviewer is outside the possible range of communication 
on the topic in question. The possibility of a complete and valid 
interview is, therefore, remote. 

The relationship between the interviewer and the respondent 
and the character of the information communicated is not, however, 
determined exclusively by the respondent’s stereotypes. There are 
a number of studies which document ilie importance of the inter- 
viewer’s attitudes and biases in determining the results of the inter- 
view. Most of these studies fail to indicate specifically the manner 
in which the interviewer bias affects intcr\ie^v results. It seems 
safe to assume, however, that, although many of these situations may 
reflect the interviewer’s failure to use tlie prescribed techniques, 
some are caused by a failure of interpersonal relations between the 
intervietver and respondent. Just as the respondent may behave 
according to his stereotyped judgments about the interviewer, the 
>nter\'icwer may be guided by his own stereotypes rather than by 
the objective characteristics of the respondent. The intcr\iewcr may. 
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lor example, set up inaccurate working hypotheses as to how the 
respondent will respond and then, without awareness, guide or dis 
tort the responses into the anticipated channels (10) 

In the two following sections of this chapter, we shall discuss 
those principles of questionnaire construction and interviewing tech 
nique which m the light of present experience are most likely 
to maximize a respondent s motivation to communicate and which 
help the interviewer to avoid the kinds of inhibiting or distorting 
lactors which have just been described 


DESIGN OF THE QUESTIONNAIRE 

Dual Purpose of the Quesiionnatre 

The questionnaire, or interview schedule, serves two major 
purposes First, it must translate the research objectives into specific 
questions, the answers to which will provide the data necessary to 
test the hypotheses or explore the area set by the research objectives 
In order to achieve this purpose, each question must convey to the 
respondent the idea or group of ideas required by the research objec 
lives and each question must obtain a response which can be 
analyzed so that the results fulfill the research objective Moreover 
the question must perform these two functions with minimal distor 
tion of the response which it elicits That is m asking a question 
of the respondent, we assume that the respondent possesses an 
attitude or opinion, or piece of knowledge Each question should, 
therefore, be constructed so as to elicit a response which accurately 
and completely reftects each respondent s position 

The second function of the questionnaire is to assist the inter 
viewer in motivating the respondent to communicate the required 
information There are many factors which determine the respond 
ents willingness to engage in an interview, as we have already 
mentioned In motivating the respondent, the skills of the inter 
viewer are of great importance, of coukc, but the questionnaire itself 
does much to determine the character of the interviewer respondent 
relationship and, consequently, the quantity and quality of the data 
collected 

Since the questionnaire is constructed on the basis of the re 
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search objectives, it is clear that constructing the questionnaire can- 
not be the first step in undertaking a research project The statement 
of the research objectives and tlie specification of the data required 
to meet those objectives must precede questionnaire construction 
The sequence of steps in planning a research project is discussed 
in some detail in Chapters 1 and 2 It will be sufficient here to pro 
vide an example of the process by which a research hypothesis 
determines questionnaire content 

Suppose that, as part of a stud) of how behav lor is influenced 
by mass persuasion, we have the hypothesis that the number of 
government savings bonds purchased is related directly to the 
amount of direct personal solicitation What data are required to 
lest this hypothesis’ What questions should be asked to elicit these 
data? 

In the present example, the investigators decided that two 
approaches should be emp!o)ed, one direct and one indirect The 
direct approach consisted of asking recent bond purchasers what 
factors had led them to buy The indirect approach during a later 
portion of the same interview led to inclusion of a number of ques 
tions concerning the respondent’s recent exposure to such influences 
as newspaper advertisements, radio, other group appeals, and m 
dividual solicitation In anal)zing the obtained data, the researchers 
sorted those respondents who were comparable with respect to 
income and other demographic characteristics into groups according 
to the frcqucnc) and t)pe of solicitation which the) had e\pcn 
TTltc bafitig bebtti tar of ebese grwapj « as eben scacited, anc! tc 
"as found tliat buying bclnvior vvas closclj related to the presence 
or absence of personal solicitation (7) 

In this example, one can sec how the qwcstionmire design flows 
logical!) from the specified research objeciivcs and must aniicipaie 
the amljsis of the data Thus, consinicuon of the qucsiionnairc 
u an imcgraietl step in getting a research project into opention 

As noted above, the second function which the qiicvnonmirc 
must perfonn is lo assist in crcaiiiig conditions under which ihc 
respondent vmII communicate full) am! free!) Research workers are 
h) no means agreed on the techniques bv v\hich this can best be 
achieved (12) A $)stcmatic methodological research on tfie inicr 
Viewer respondent relationship has rcccntl) been completed b) the 
National Opinion Research Center, and the final report is now in 
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the process of publication (10) The experience ol the present writers 
argues that the characteristic of respondent orientation is of 
primary importance in maximizing communication The concept 
of respondent orientation as a characteristic of the questionnaire 
has an obvious analogy in the clinical concept of client centeredness 
(17) The comparison is more than a superficial one The question 
naire makes use of some of the techniques and to a small degree 
serves some of the purposes of client centered therapy The differ 
ences between the research and the therapeutic interview are many 
of course In the research interview content is determined primarily 
by research objectives rather than by the respondents needs simi 
larly the pace and sequence of questions is for the most part beyond 
the control of either the respondent or the interviewer Even the 
decision to engage in the interview is not wholly of the respondent s 
volition In spite of these limitations there are a number of respects 
in which the interview may be oriented toward the respondent and 
the questionnaire so constructed that the needs and reactions of the 
respondent are taken into account 

The preceding pages have presented the major criteria by which 
a questionnaire might be judged Howeter valid these criteria may 
be they do not solve the specific problems of question wording and 
question sequence which confront every social scientist who uses the 
interview The remainder of this section is devoted to a discussion 
of just such specifics a discussion which attempts to develop the 
dos and donts of questionnaire construction out of the major 
purposes which the questionnaire must serve The topics included 
by no means exhaust the subject More detailed treatments have 
been made by Payne (16) Parten (15) Gantnl (6) Blankenship 
(5) and others The problems discussed here are those which ap 
peared most relevant in terms of the criteria cited and therefore 
most important in creating an adequate instrument for collecting 
social data by means of interviews 


Language 

In the construction of a questionnaire the primary criterion 
for the choice of language is that the vocabulary and syntax should 
offer maximum opportunity for complete and accurate communica 
non of ideas between intervietver and respondent Not only should 
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the words chosen be within range of the respondents \ocabuIary, 
but also his colloquialisms and cliches should either be known and 
used meaningfully or avoided Most simply stated, the language of 
the questionnaire must approximate the language of the respondent 
In most cross section surveys, one strives for simplicity, and properly 
so For a study of physicians or lawyers, however, different and 
specialized vocabularies would be more appropriate To the extent 
that the respondent population is not homogeneous, compromise 
IS unavoidable In such cases the solution of the problem consists in 
using language which communicates successfully to the least sophis 
ticated of the respondent population and, at the same time avoids 
the appearance of oversimplification 


Frame of Reference 

To say that a questionnaire should be cast m the language of 
the respondent is relatively unequivocal and straightforward It is 
equally important, and considerably more difficult, however, to 
phrase questions which take account of the frame of reference 
which respondents bring to the subject under discussion Neverthe 
less, the questionnaire must introduce each topic in a form which 
ties into the perceptions of the respondent and is consistent with 
the respondent’s notions of what is and is not salient to the topic 
under discussion The development of a topic from one question 
to another must not only meet the researcher s criteria for reason 
ableness and logic, it must also meet those of the respondent Thus, 
frame of reference becomes another dimension in which the re 
searcher must begin at the point where the respondent is —it 
tnust be respondent oriented 

Bancroft and Welch (2) present an example of the effect of the 
respondent s frame of reference on his replies They found that the 
senes of questions used by the Bureau of the Census to ascertain 
the number of people in the labor market consistently underesti 
tnated the number of employed persons ^Vhen asked the question 
“Did you do any work for pay or profit last week? respondents 
reported what they considered to be their major activity Young 
people attending college considered themsehes to be studcnis, even 
if they were also employed on a part time basis 'Women who cooked, 
cleaned house, and raised cliildren spoke of themsehes as house 
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the process of publication (10) The experience ot the present writers 
argues that the characteristic of respondent orientation' is of 
primary importance m maximizing communication The concept 
of respondent orientation as a characteristic of the questionnaire 
has an obvious analogy in the clinical concept of client centeredness 
(17) The comparison is more than a superficial one The question 
naire makes use of some of the techniques and to a small degree 
serves some of the purposes of client centered therapy The differ 
ences between the research and the therapeutic interview are many, 
of course In the research interview, content is determined primarily 
by research objectives rather than by the respondent’s needs simi 
larly, the pace and sequence of questions is for the most part beyond 
the control of either the respondent or the interviewer Even the 
decision to engage in the interview is not wholly of the respondent’s 
volition In spite of these limitations, there are a number of respects 
m which the interview may be oriented toward the respondent, and 
the questionnaire so constructed that the needs and reactions of the 
respondent are taken into account 

The preceding pages have presented the major criteria by which 
a questionnaire might be judged However valid these criteria may 
be, they do not solve the specific problems of question wording and 
question sequence which confront every social scientist who uses the 
interview The remainder of this section is devoted to a discussion 
of just such specifics a discussion which attempts to develop the 
’dos and don’t s of questionnaire construction out of the major 
purposes which the questionnaiie must serve The topics included 
by no means exhaust the subject More detailed treatments have 
been made by Payne (16), Parten (15) Cantril (6). Blankenship 
(5), and others The problems discussed here are those which ap 
peared most relevant m terms of the criteria cited and. therefore, 
most important m creating an adequate instrument for collecting 
social data by means of interviews 

Language 

In the construction of a questionnaire, the primary criterion 
for the choice of language is that the vocabulary and syntax should 
offer maximum opportunity for complete and accurate communica 
tion of ideas betiveen intervieiver and lespondent Not only should 
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a question, there is an implication that the respondent should be 
in possession of an adequate answer and that if he cannot ans\\ er, 
he IS somehow discredited If, for example, in a questionnaire deal 
ing with public attitudes toward problems of atomic energy, an 
interviewer asks a respondent, * What precautions are appropriate 
for a technician handling radioactive isotopes? an immediate and 
very common respondent reaction will be embarrassment and resent 
ment at being asked a question which he is unable to understand 
Not only will the researcher have lost the answer to a question, 
but he will also pay a heavy price in terms of decreased motiva 
tion to communicate Another possibility, of course, is that the 
respondent, feeling obligated to show his knowledge xvill pretend 
to competence xvhich he does not possess 

The importance of asking questions appropriate to the respond 
ent’s level of information, and not productive of respondent embar 
rassment, does not necessarily limit us to asking questions to which 
every respondent knows the answer It does mean, howexer that 
caution in wording questions must be used when we anticipate that 
a considerable proportion of respondents will not be m possession 
of the answer For example, the question quoted abo\e might be 
preceded by a statement such as "Nfany people haxen’t had an oppor 
tunity to learn a great deal about the technical problems of handling 
atomic material, but some have picked up information on this 
subject Do you happen to know ? ' 

This problem is sometimes referred to as expert error— that is, 
the error of ascribing to the respondent a degree of expertness m a 
particular field which he docs not actually possess These 'expert” 
questions mi) require the respondent to engage in uncomfortable 
self analysis, to be xerbal about material avhich is really unanalyzed 
or unxerbahzed and therefore not consciously axailable Suppose 
uc ask an industrial employee, 'What is the state of your morale, 
and xvhat is the reason for your feeling that x\ay^ ’ It xxould be as if 
a doctor asked a patient for the name and cause of his disease, rather 
than asking for the patient’s symptoms from uhrch tfie mtare of the 
disease may be inferred 


foetal Acceptance 

Another cliaractcrisuc of ihe respondent centered questionnaire 
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wives esen lE they also did some work for pay outside the home 
The effect of the respondents frame of reference was to classify as 
nonworkers many thousands of people who met the census definition 
of workers The solution involved revising the sequence of interview 
questions beginning with the acceptance of the respondent s classi 
fication of himself Thus, people tvere asked first what their major 
activity was Then, those people who gave nonworker’ responses 
were asked whether, in addition to attending school or keeping 
house, they did any work for pay The effect of this change was to 
raise the official estimate of employment by more than one million 
persons 

The respondent s frame of reference may also be important 
in determining whether he will be willing to communicate a given 
piece of information He may be reluctant to communicate if he fails 
to see the relationship between a question and his own perception of 
the research objectives Thus a survey respondent who has talked 
freely about foreign policy may suddenly balk at being asked his 
age or education Although these are unlikely to be threatening 
questions, they do not fit his perceptions of the research needs The 
collection of data on family income cued earlier m this chapter, 
provides another example of the extent to which respondent be 
havior may depend upon his perception of what is relevant The 
collection of detailed data on personal income, unsuccessfully 
attempted m many early studies was achieved by introducing a 
request for income data as part of a program to assess, and perhaps 
ultimately to solve problems of consumer credit, spending, and 
saving In the context of discussing savings, plans for consumer pur 
chases, and attitudes and expectations about economic and personal 
financial status, the question of family income appears to the 
respondent as reasonable and relevant 


7n/ormofjon Level 

A question must be worded so that it ties into the respondent's 
present level of information in a meaningful way No unrealistic 
assumptions should be made about the experiness of the respondent 
or the amount of information he possesses The importance of this 
rule for questionnaire construction lies m the fact that when the 
interviewer, with the authority of his role, asks the respondent 
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merely agreeing \uth the language of the question It is more 
difficult to respond, I disagree/* since this response seems to contra 
diet the interviewer, or at least goes counter to the ideas of the 
peison who worded the question 

One way in which a question may suggest a positive or negative 
answer is through the use of words which have become emotionally 
loaded,” either favorably or unfavorably In our culture there are 
many words so affect laden that it is virtually impossible to expect 
a respondent to gue a response to the concept behind these words 
For example, prior to World War II, Nazi ’ had already become 
an emotionally colored word As a result, one obtained very differ 
ent responses to a question referring to Nazi Germany,’ rather 
than simply to “Germany *’ 

Another way in which a question may encourage a particular 
response is by associating one of the aUernaiive responses with a 
goal so desirable that it can scarcely be denied Thus, the question 
Do you favor or oppose higher taxes to prepare for the dangers 
of war? ' associates higher taxes with defense against attack and 
implies that a negative answer reflects indifference to the memce 
of attack Even if the respondent is permitted an unstructured reply, 
the question does much to bias his anstver If he is gi\en only the 
alternatives of acceptance or rejection, as in the topical public 
opinion poll, the biasing effect of the question is even more serious 

A loaded question is not necessarily undesirable and often 
has a real place m the questionnane The problem is to a\oid load 
ing if one is looking for an undistorted response The folloising is 
an example of a strongly loaded question svhich was purposely used 
in one study ‘^\ould >ou Ea\or sending food oserseas to feed the 
starving people of India? In this case the question followed a 
series of unloaded questions and was used to determine the number 
of people who were so strongly against shipping food tint they 
rejected the idea in spite of the siioiig emotion il context of snr\ ing 
people " 


The Single Idea 

Questions should be limited to a single idea or to a single 
reference The problems encountered in this area are illusinicd 
in the following question ' Do >ou fa\or or oppose unemployment 
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15 Its emphasis on the acceptability of a wide range of responses 
No question should confront the respondent with the necessity 
of giving a socially unacceptable response If we expect the respond 
ent to answer freely and spontaneously we must help him to feel 
that the entire range of possible responses is acceptable— acceptable 
not only to the interviewer but also in terms of the respondent s own 
standards for himself For example if after a presidential election 
we wish to ascertain who in the population did and who did not 
vote we find ourselves in the position of asking respondents about 
a situation m which thev may perceive only one socially acceptable 
alternative The civic minded responsible citizen voted therefore 
the respondent voted or at least should have voted and does not 
wish to be put m the position of telling the interviewer that he failed 
to do so This hesitancy can be overcome at least in part through 
question wording For example some form such as the following 
might be used You know in the last election about half of the 
qualified voters actually got to the polls and about half were unable 
to did you happen to vote? 

Offering a range of responses which meets the respondents 
criteria of social acceptability is necessary to good question formula 
tion A broader statement might be that the question must nevei 
constitute a threat to the respondent s ego Such a threat may be 
introduced if the respondent is required to give an answer which 
he feels is socially unacceptable or it might come about if the 
respondent is placed in a position where he feels less vsell informed 
than he should be 


Leading Qi estions 

Questvons should be phrased so that they contain no suggestion 
as to the most appropriate response For example a question de 
signed to elicit general attitudes toward rent control might read 
How do you feel about rent control? A form of the same question 
which IS obviously biased might be You wouldn t say that you 
were in favor of rent control would you’ This kind of bias is so 
easily recognized that we avoid it almost without effort A more 
subtle form of biased wording might be Would you say that you 
are m favor of rent control’ This question makes it easier for the 
respondent to answer yes than no In answering yes he is 
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sequence of questions the frame of reference is gradually nai rowed 
by asking more specific questions The purpose of the funnel 
sequence is to prevent early questions from conditioning those 
which come later and to ascertain from the first open questions 
something about the respondent's frame of reference The following 
series of questions illustrates the funnel approach 

Question 1 How do you think this country is getting along in 
Its relations with other countries? 

Question 2 How do you think we are doing in our relations 
with Russia’ 

Question 3 Do you think we ought to be dealing with Russia 
differently than we are now? 

Question 4 {If yes) What should we be doing differently? 

Question 5 Some people say wc should get tougher with Russia 
and others think we are too tough as it is how do 
you feel about it’ 

The reader will notice that the first question is very general in 
Us approach It does not establish a frame of reference— a trend of 
thought— m regard to the country under discussion in terms of 
diplomatic relations or of economic relations It permits the re 
spondent great freedom in discussing the topic From the answei 
fo the first question, we can probably infer the frame of reference 
of the respondent In the second question we have restricted the 
area to one country, Russia The third question is aimed at the 
I'espondents opinion of how the United States ought to deal with 
Russia and the fifth becomes very specific by asking whether we 
should exert more pressure or be more lenient If for example 
Question 5 had been asked any earlier m the sequence, it might 
'veil have conditioned the answers to tfie other questions Tiie 
funnel technique is, therefore, often very Iielpful in avoiding the 
distortion of a question by those which precede it It enables us to 
analyze tlie frame of reference which the respondent is taking and 
It enables us to get a general aflcctivc response before pinning the 
person down to more specific points 

The first two or tliree questions in a questionnaire often In\e 
a dual function On tJie one hind they are included to obtain 
information on specific research objectives, but they also help to 
educate and motivate the respondent In many instances the re 
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insurance and pension plans?" Many answers to this question would 
not permit the researcher to determine whether the respondent 
is answering one or both of the items mentioned in the question. 
The most acceptable formulation of a question in this area would 
depend to some extent on the specificity of the research objectives. 
If the purpose is to find out the respondent’s attitude toward pen- 
sions and unemployment insurance specifically, it would be neces- 
sary to ask two questions, one referring to each of the two proposals. 
If, on the other hand, the purpose of the question is to get some 
notion of the respondent’s general attitude in the area of worker 
benefits, it may be possible to ask a global question such as "Do you 
favor or oppose such worker benefits as unemployment insurance, 
pension plans, and the like?” It must be kept in mind, hosvever, 
that if a global question of the latter type is asked, the interpreta- 
tion must be very conservative. In other words, a positive response 
to a global question must be taken only to indicate favorableness 
in the general area and cannot be interpreted as indicating respond- 
ent support for any of the specific examples cited. 

Question Sequence 

Aside from the wording of individual questions, the researcher 
needs to give thought to the arrangement of the questions in a 
questionnaire. At several points in this chapter we have discussed 
the concept of respondent orientation, which is also important in 
considering question sequence. Thus, questions should be so ar- 
ranged that they make the most sense to the respondent— that is, 
the sequence of ideas in a questionnaire should follow the logic 
of the respondent. It may be that questions which are associated 
together from the analyst’s point of view are widely separated in 
the questionnaire. The sequence of questions should be determined 
primarily by the interview process rather than the research process. 
A well-designed questionnaire facilitates the easy progress of the 
respondent from item to item and often leads him to anticipate the 
next question because it seems to him the logical topic to discuss. 

The sequence of questions may also be determined by what is 
called the “funnel approach." This refers to a procedure of asking 
the most general or the most unrestricted question first and follow- 
ing it with successively more restricted questions. Thus, in the 
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contained in the question so that tlie respondent meiel) has to 
select the category which comes closest to his position An example 
of a closed question is, ‘Do you think your income ^\lII be higher, 
lower, or about the same this year as it was last year? 

Generally speaking, the closed question is well adapted to situ 
ations m which (I) there is only one frame of reference from which 
the respondent can answer the question (2) within this single frame 
of reference, there is a known range of possible responses and 
(3) within this range, there are clearly defined choice points which 
accurately represent the position of each respondent Two examples 
Will help to clarify these points The first is the classification of 
respondents by marital status In this case, there is a known range 
of possible responses a person is either single, married, divorced, 
separated, or widowed Within this range, the choices are clear and 
the question has but one frame of reference for all respondents 
Here the closed question is desirable and can be worded Are you 
Single, married, divorced, separated, or widowed? The respondent 
merely has to select the response which defines his marital status 

Another example of the closed type is the question Would 
you say your present income was higher, lower, or about the same 
as your last year's income? In this question the respondent is asked 
to compare two facts which are known to him The frame of ref 
erence is limited to an income comparison for two years, and the 
choices are clear 

Crutchfield and Gordon Iiave provided an excellent documen 
tation of the effects of using the closed question improperly (8) 
The following question was asked on a national survey After the 
war would you like to see many changes or reforms made in the 
United States or would you rather have the country remain pretty 
much as it was before the war?' The ansivers indicated that the 
majority of people wanted things to remain as they were A folloiv up 
was made in which the same question was asked and nondirective 
probes were then used to ascertain what the respondents \scrc 
concerned with when ansv,enng the question The responses showed 
that respondents ansivered from sc'en frames of reference Some 
Were concerned with domestic issues (employment conditions, stand 
ard of living, etc), some with tcclmical improvements (better 
transportation, communications, etc), others with political affairs, 
and so on Since the original resctrclicr was unaware of the varying 
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spondent may not know the kind oI responses expected of him- 
that is, whether a one-tvord answer will suffice or whether he is 
being asked to discuss the subject in detail. He may wonder about 
being cross-examined, or he may be simply confused about the 
interview demands regardless of the initial instructions. During the 
first two or three questions the interv'ieiver, by his probing, his reac- 
tions to responses, and his general behavior toward the respondent, 
educates the respondent in the role which is expected of him in the 
interview. 

In addition to their orienting or educational purpose, the first 
questions also serve to motivate the respondent to participate more 
thoroughly by involving him in the topic under consideration. The 
first few questions may, in fact, set the tone of the entire interview. 

Often a questionnaire covers more than one general topic. This 
may create difficulties in the intcrs'iew, since the respondent has to 
be helped to change his frame of reference from one topic to the 
next. One efficient way to help the respondent orient himself to a 
new area of discussion involves the use of transition statements oi 
transition questions. Such a statement might be, ’'Well, we’ve been 
discussing our relations with the Far East; now we want to talk a bit 
about the way things are going for us in Europe. How do you feel 
about our relations with the countries in Europe?” Statements of this 
sort help the respondent to shift gears and to transfer his attention 
to a new area of discussion. 


The Form of the Question 

Thus far we have discussed the wording of questions without 
considering the problem o£ the form of the response— that is, whether 
the respondent is to reply in his own words or whether he is to 
select from a series of preassigned categories the response coming 
closest to his own opinion. Questions of the former type are termed 
open or unrestricted ; the latter type of question is “restricted” 
or “closed." The open question is one in which the topic is struc- 
tured for the respondent but he is given the task of answering in 
his own words, structuring his answer as he sees fit, and speaking 
at whatever length he desires. An example of an open question is, 
"How do you feel about Negroes and whites working together in 
this factoiy?” In the closed question the possible responses are 
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encountered in coding responses In both types the respondent s 
attitude or perception must be categorized If the question is closed, 
the categorization is made by the respondent or the interviewer 
If the question is open, the categorization may be made by the 
interviewer, it is usually considered preferable, however, to have 
the responses coded at some central place by people trained for 
this work Each of these coding procedures has advantages and 
disadvantages, which are discussed and evaluated in Chapter 10 
For a more detailed discussion of open and closed questions and 
their respective uses, the reader should see Lazarsfeld (13) Lazars 
feld contends that the methods can be combined effectively if the 
open question is used in an elaborate pretest followed by closed 
questions in the main study with open questions used in a follow up 
of critical cases 


T/ie Pretest 

No matter how astute the researcher has been in \vording his 
questions and designing his questionnaire, he needs to try them out 
With respondents before launc^?lng into the actual field studies The 
pretest is, in a sense, a miniature study in itself The first function 
of the pretest is aimed at testing the questionnaire from the research 
point of view The interviews should be analyzed to see ivhether 
the responses fulfill the research objectives Some of the researcher’s 
‘ best questions” often fail to elicit the type of response which meets 
the objectives An analysis of these trial interviews in relation to 
the objectives will increase the probability of fulfilling the research 
objectives Often the pretest calls for major revision of the questions, 
and several pretests are required until a workable questionnaire is 
achieved 

A second objective of the pretest is to determine the extent to 
which the questionnaire meets the criterion of respondent orienta 
fton m all its aspects Does the questionnaire promote the appro 
priate relationship with respondents? Do respondents understand 
the questions'^ Can the questions be asked without having to be 
explained or reworded? There arc no exact tests for these clnmc 
teristics The help of experienced interMCucrs is most useful at tins 
point m obtaining subjectne evaluations of the questionnaire 
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frames of reference, his interpretation of the findings was quite m 
error This illustrates the danger of using closed questions when 
more than one frame of reference is possible 

Let us consider another example At the present time, do 
you think personal income taxes arc too high too lou, or about 
right? The alternatives within the question are incomplete and 
fail to allow for the person who feels that the income taxes are fair 
at certain income levels but are unfair at other leiels Neither are 
the alternatives adequate for the person who feels that taxes are too 
high because of heavy government expenditures but that if the 
expenditures continue then taxes must remain high Thus, for 
many people the alternatives presented do not include choice points 
which closely approximate their attitudes Such people are forced 
either into discussing the topic more fully with the interviewer or 
into selecting a category which is a poor approximation of their 
position 

The open question has many advantages stemming from the 
fact that the respondent is encouraged to structure his answer as he 
wishes The technique provides a means of obtaining information 
which cannot be obtained adequately by use of a closed question 
For example u permits the respondent to slate his otm frame of 
reference when this is desirable The potentialities of the open 
question for discovering motivation have been ingeniously explored 
by Lararsfeld (14) 

Another advantage of the open question 15 the information 
which the answers indicate with respect to the respondent s level 
of knowledge or degree of experlness If the respondent has been 
led to discuss his opinions of the Atlantic Charter, one is able to 
analyze not only his attitude but also his level of information 

The relatively free interchange between interviewer and re 
spondent which is characteristic of the open question permits thf 
interviewer to discover whether die respondent clearly understands 
the question which is being asked of him On the other hand, once 
the respondent has selected one of the proffered alternatives to a 
closed question the interviewer can assume only that the respond 
ent understood the question and chose the alternative which best 
approximated his own position 

Another difference between the open and closed question is 
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situation only in the absence of certain specific kinds of barneis to 
communication 

When the interviewer first faces a respondent, he finds that the 
relationship has some structure even before a word is spoken On 
the one hand, the respondent probably will be polite enough to let 
him talk, on the other hand, there may already exist certain barriers 
which must be overcome For example, the public opinion inter 
viewer is frequently mistaken for a salesman by the respondent 
Another barrier to communication arises from the respondent s 
perception that granting the interview will in some way make him 
vulnerable This is essentially a problem m reassuring the respond 
ent with respect to the anonymous or confidential nature of the 
interview A third barrier comes from a rather frequent respondent 
perception that the interview may be intended in some subtle way 
to check up on him or his activities Handling this sort of problem 
calls for a convincing explanation by the interviewer of the purpose 
of the study and particularly of the method by which the respondent 
was selected 

The positive motivation in terms of the respondents goals 
comes from a careful statement of the purpose of the research The 
interviewer tries to sense the respondents wishes or goals ivith 
respect to the interview process and having appraised these to 
explain to the respondent how the interview relates to them For 
example, an interviewer working on a study of public attitudes 
toward current matters of foreign policy might come upon a re 
spondent who on hearing the purpose of the survey, says to the 
interviewer You don t want to talk to me about foreign policy 
What I think about those fellows in the State Department uouJd 
curl their hair You d better find somebody who is a more agreeable 
type The interviewer would assure the respondent tint the pur 
pose of this study ts not to find out simply the opinions of people 
who are endorsing current foreign policies He ivould emphasize 
that the interview provides an opportunity for the respondent to 
register his criticism in a place where it might have some effect on 
public officials who sincerely want to learn the general public atii 
tude, ivhetlier it is critical or appreciatiie 

In some researches the respondent goal is rather cleirl) per 
ecued, such as in the case of tlic worker who is asked to participate 
in a study which may result m better working conditions or higher 
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principles of interviewing 

The preceding section dealt with the instruments of data col 
lection This section will discuss the specific techniques which the 
research interviewer uses The techniques proposed are a systematic, 
well tested set of procedures which are consistent with the principles 
of communication discussed earlier m this chapter 


The Inlro&ucUon to the Interview 

The first step is often the most difficult for the interviewer, 
because at the initial contact the respondent must be motivated to 
permit the interview Ordinarily the interviewer will follow a 
sequence of procedures approximately as follows 

1 Explain the purpose and objectives of the research 

2 Describe the method by which the respondent was selected 

3 Identify the sponsor or the agency conducting the research 

4 State the anonymous or confidential nature of the interview 

In the early phases of the interview, the interviewer plays one 
of his most important and one of his most autonomous roles It is 
difficult to describe the precise acts which an interviewer should 
perform m order to provide adequate motivational basis for the 
respondent to communicate the information which he seeks The 
establishing of rapport is clearly not a scientific procedure in the 
sense of being capable of objective statement It is rather a skill 
which, depends primarily on the know how experience and sensi 
tivity of the interviewer It is ihis function of the interviewer ivhich 
makes great demands on the qualities of clinical insight and 
intuition 

We have already mentioned that the forces leading a respondent 
to communicate can be thought of m terms of a means end or 
path goal sequence in which the respondent gives information be 
cause he sees the information giving process as a means of attaining 
some goal which he considers desirable Secondly the respondent 
IS motivated to give accurate and complete information as a means 
of attaining some satisfaction out of the relationship with the inter 
viewer Thirdly the respondent communicates m the interview 
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pay In other studies the respondent goal is more obscure In a 

laboratory research problem lor example, the 

gam only the prestige o! partiapating in a scientific or officia 

^"'*'^Another motivation the interviewer should tap comes from the 
personal relationship which he builds with the respondent In part 
this relationship depends upon the interviewers being perceived 
as a desired agent of communication or change However, as m 
the therapeutic interview the qualities of acceptance understand 
mg and receptivity seem to have inherent values for the respondent 
Some evidence for the importance of the interviewer respondent 
relationship was obtained by the Survey Research Center when 
respondents who had been interviewed about their incomes, savings 
and buying plans were polled by mail about their reactions to the 
interview Their replies were more often couched in terms of the per 
sonal relationship and personal qualities of the interviewer than in 
terms of the content of the study or the apparent purpose of the 
inquiry Typical comments mentioned the fact that the interviewer 
ivas a very understanding person or that the interviewer had a keen 
insight into the respondent s situation 

Often the interviewers contribution to respondent motivation 
IS referred to as rapport The term has come into common use 
and indicates an increasing sensitivity on the part of researchers 
to the importance of the interviewer respondent relationship At 
times the use of the term suggests a superficial approach to respond 
ent motivation Thus rapport is referred to as if it were some 
tangible quantity or some specific task which was to be gotten out 
of the way early in the interview as a preamble to getting on with 
the mam business of data collection There is the implication that 
after the interviewer has said Good morning and inquired after 
the health of the respondents family with the properly solicitous 
inflection he can ignore the relationship with the person giving 
him data Contrary to the implications of this approach rapport 
IS not something which is plugged m at the beginning of the 
interview m order to get it off to a good start Rapport refers to 
the atmosphere or climate of the entire relationship between re 
spondent and interviewer 

Although rapport or the climate o£ the interviewer respondent 
lelationship has yet to be reduced to quantifiable factors we can 
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following types of situations (1) to elicit additional information 
from the respondent when further information is necessary to the 
research objective, and (2) to clarify or make more speafic informa 
tion which the respondent has already given All of this must be 
accomplished without changing or biasing the data 

The "probing” techniques useful for such purposes can be 
classified generally as "nondirective ” They enable the interviewer 
to act as a catalyst— that is, to bring about a reaction without him 
self becoming part of the reaction The effect of such probing is to 
increase the intensity or "response getting * poiver of the stimulus 
question without changing its content or structure 

Thus, to gam more information the interviewer uses such 
phrases as "Would you tell me some more about that^ ' I'm inter 
ested m what you’re saying Could you give me a little more about 
that? ’ or "I see what you mean Can you tell me a little bit more 
about how you feel there?" These statements indicate that the 
interviewer is interested, understands what the respondent is saying, 
and IS making a direct bid for more information To accomplish 
the second task, that of clarification of information already given, 
the interviewer might use such probes as "Now let me see if I have 
It straight As I get it, you feel ' ’ and then summarize what the 
respondent has said Or he might say, "I would like to read my notes 
back to you to see whether I have your point of view straight ' 

It IS through the use of such nondirective probes ihit the 
interviewer does much to develop the permissiveness and warmth 
which are so important in the interview The reader who is familiar 
wtfi the literature on client centered counseling will recall the 
stress which is placed on the atmosphere of permissiveness as the 
basis for permitting the client to examine his own attitudes Such 
atmosphere permits the client to verbilize the deeper attitudes 
"Inch are usually concealed from outsiders Many of the same 
d>nanucs are present at a more superficial level in the researcli 
mterview, whether one is dealing with personal attitudes or factual 
data Let us consider some examples of the effect of skillful probing 

I Hou do jou feel about sending mone) nnti lielp to other 
countries’ 

R Well, I don’t know Sometimes I think we go too far 
I I sec Cm jou tell me i little more about wlnt )oii Insc in 
mind’ 
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It has the same psychological impact for each Since, therefore, we 
cannot tailor the question lor each respondent, the best approxi 
mation to a standard stimulus is to word the question at a level 
which IS understandable to all respondents and then to ask the 
question of each respondent m identical fashion This, then, is 
the function of the interviewer in using the questionnaire as the 
stimulus The only instance m which the interviewer is permitted 
to vary this procedure is when an individual is unable to under* 
stand the question as worded Even in such cases, the interviewer 
IS encouraged to repeat the question verbatim before explaining it 
In many cases, apparent lack of understanding is a matter of atten- 
tion fluctuation rather than inability to grasp the question meaning 
In such instances, a simple repetition of the question will suffice 
Except for these minor variations, the interviewer’s role with 
respect to the questionnaire is to treat it as a scientific insirument 
designed to administer a constant stimulus to a population of re 
spondents This technique is necessary when quantifiable data are 
desired In some research of an exploratory nature, or where sub 
jective analysis is contemplated, the interviewer ma^ be permitted 
much more leeway m the use of the questionnaire In some research 
he may tailor his questions to each respondent, with the researcher 
indicating only the areas to be investigated Where quantifiable 
data are needed, however, the more rigid use of the questionnaire 
appears necessary 


Sttmulatmg Complete Responses 

In many cases the use of the question evokes a response which 
is incomplete or which is unclear The interviewer must have some 
technique which wiU enable bim to stimulate the respondent to 
further verbalization Moreover, he must achieve this without sacn 
fievng standardization Fox example, if a question is asked of all 
respondents, we have, so far, comparability If at this point each 
interviewer asks a different subquestion which he makes spontane 
ously, the responses are no longer responses to the original question 
but will vary from interviewer to interviewer depending upon the 
subquestion which was asked This defeats the objective of stand 
ardization 

Specifically, the interviewer needs techniques to handle the 
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having served to make the point was dropped The final answer of 
400 bushels was almost certainly closer to the actual fact 

As indicated earlier in this chapter the general effect of this 
type of interpersonal relationship is pleasurable for the respondent 
in that he has the opportunity to talk with a skillful interviewer 
He reacts to the permissive, accepting atmosphere by communicating 
Willingly with the interviewer 


Recording the Responses 

One final job remains for the interviewer this is to get a faith 
ful and accurate report of the responses Experience has shown 
that the only accurate way to reproduce the responses is to record 
them during the time of the interview either by 
ods or by having the interviewer take extensive notes A tol 
of relevant inforaation is almost certain to be lost if the recording 
IS left until the interview has been completed It is no wi 
scope of this chapter to discuss various kinds of recording devices 
Whatever the method, however, the tnterviewer must be trained 
in Its use and it must be carried out faithfully during the process 
o£ the interview itsel£ 


SAMPLE INTERVIEW 

In order to demonstrate some of the ‘-^^-uques which have 
been discussed in this chapter, we have included a bnef 
of a data collecting interview taken in an in ustria p 
factoring tractors The respondent was a foreman The example is 
an excerpt from a phonograph.cally recorded interview I' hastee^ 
edited slightly in places to make it more intelligi 
viewers questions preceded by an asterisk are ques lo 
All others are the interviewers probe This 
as an ideal interview but merely to demonstrate the techniques 
Ijy one experienced interviewer 

I 1 ‘What do you do on your job? i ^ tr^neral tie 

1 The objective is to obtain a general pic 
tur, o! the f>pe of wor/ ood r»po,mb,Ut,r, 
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R Well majbe we ought to give some help but my gosh when 1 
see our tax money go to help some ot those countries who 
arent doing much tor themselves 1 thinV sometimes well 
better lay off 

I Sometimes you feel we ought not to help them 

R Thats rightl I think wed better let them go their own way 
and to hell with theml 

In this example, the respondent made a mildly critical state 
ment at first The interviewer reacted to this by being nonevaluative 
and yet accepting He didn t criticize the respondent nor did he 
agree with him he merely indicated a general acceptance of the 
statement The result of this was a somewhat more pointedly critical 
statement The nonevaluative acceptance by the interviewer per 
mitted the respondent to make his final bitter response without 
feeling the need to defend or modify it 

The next example is taken from an interview with a farmer 
The interview was concerned almost exclusively with problems of 
farm production 

I How many bushels of wheat did you harvest last year? 

R My gosh we had a tembU yearl When wed ought to be 
planting last spring it rained all the lime and then it got dry 
and everything burned up We didn t get more than 300 
bushell 

I 1 see Well you said you didn t get more than 300 bushels 
Can you give me a little closer estimate? 

R Well like I said it was an awful year around here but I 
guess we got a little more than SOO-between 350 and 400 I 
guess actually 

I 350 to 400 you say Which would be closest? 

R Oh I think we estimated u at right around 400 bushels 

Notice that the interviewer again began with a nonevaluative 
statement essentially repeating the respondents first estimate The 
effect was that the respondent revised his estimate of wheat yield 
It seems apparent that his first response was more concerned with 
the misfortunes of the aop than with a precise estimate of it The 
interviewer ignored the attitude and focused on the factual part 
of the response The result was that the estimate of 300 bushels 
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formation already obtained and then a vntnal 
verbatim lepeat of the original question 
R: Well, the track comes down the assembly 
line and the tractor comes down the assem 
bly line and the last thing that we do to it is 
to put these steel tracks on it so that it can 
be driven away and, uh, we've got an electric 
hoist that lifts the heavy tracks and puts ’em 
m; we jockey ’em into place and then, uh, 
some of the men work on the top of the track 
fastening cleats to it, and some of the men 
work on the bottom and, uh, I kind of look 
to see that they’re doin’ it all right and help 
em out if there’s any trouble (Pausef—l^eep 
track of our production 

5- 1: So, one of your jobs as foreman is to see 
that the men are doing their work properly 
as you indicate and, uh, keep track of pro 
duction. 

5. This illustrates the use of a content sum* 
mary as a probing technique The interviewer 
merely summed up the statements which xeere 
made. This device is particularly effective after a 
rambling, incoherent statement The summary 
serves to focus attention on the central content 
of what has been said In addition, it indicates to 
the respondent that be has been communicating 
ideas and that the interviewer has accepted the 
ideas. Usually, the summary stimulates further 
responses, either additional data or clarification 
of what has been reported pinnously 
Rt Yes, that’s right (Pause) and tlien ever) 
day they send me a report of what our nojk 
"“as like the day before, hou much work we 
sot out and how much scrap there nas, and 
uh— it's up to me to see that the amount of 
"ork is okay and that there is not too much 
scrap. 
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The question utas asked u/ord for wotd as il 
appeared on the questionnaire 
R Well, 1 m a track foreman, that is, I’m in 
charge of the men who are putting together 
these metal tracks, you know, that tractors 
run on 

2 I The tractors run on, you say? 

2 l/n/orfunot^fy a Ironscnfifion does not 
show tnfiecttons and emphasis In this case the 
interviewer's question had a slight rising inflec- 
tion on the end, indicating a mild ‘T don't quite 
get you" probe This probe does not really lead 
in the direction of the objective of the question, 
but It does give the interviewer a better back 
ground for further answers by giving him 
information about the respondent’s work 
R Yes, these big heavy tractors run on a 
steel track like a tank and, uh, after the trac 
tors been assembled, weve gotta hang one 
of those heavy steel tracks on each side of 
them 

3 r Isee 

3 These brief, permissive, encouraging com- 
ments aJifieoT fyeguenly throughout the inter 
view This type of comment and nodding of the 
head, indicating and encouraging comments, are 
the most frequent "technique ’ which the inter 
•uiewer uses In this recording many are lost m 
the reproduction 

R I’m in charge of the crew that does that 

4 / Well, will you tell me a Uttlc bit more 
about your job— you say you’re m charge of 
the crew, just what kind of things do you do? 

4 This serves to bring the respondent back 
info the area of the question objective The reader 
will note the brief summary of the pertinent in 
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11 / I see Well, that gives me an idea of what 
your job is and how long you’ve been on the 
job *Now tell me, how do you feel about 
the job you have now? 

11 This illustrates a brief transition state 
merit The interviewer by kts remarks indicates 
that one area of the questionnaire is completed 
and another is to be introduced This is a useful 
technique to help the respondent shift his frame 
of reference to the new topic 

•R Well, it’s better than when I was on the 
track line 

12 I How’s that? 

12 Another probe which is used when clan 
fication is wanted The inflection indicates ' Vm 
not quite sure / understand you Will you amplify 
that remark^' 

R Well, for one thing, foreman's pay is 
higher and also it’s more regular— it goes 
right on, It’s not hourly 

13 I Uh hum 

^ And, uh, besides that, I like the kind of 
work better 

1^ f Uh hum— okay, you say you like the job 
better than the one you had before that, uh, 
let’s take it all in all, how do you feci about 
your job? 

14 Up to tins point the respondent has 
been answering tn terms of details on hts job 
The objective calls for general affect The inter 
viewer tries (o communicate this frame of refer 
ence tn this way of re asking the question stressing 
the over all aspects 

•R Well, I guess I like it all right— it’s got 
*ts headaches like all good jobs do, I guess 

15 I Well, that’s one thing we want lo talk 
sbout, uh, you’\e giNcn me some information 
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/ I see, uh, are there any other kinds of 
things you do on your ]ob? 

6 This IS a very direct bid tor additional 
information The stress on the word "kinds"’ is 
directive in that it asks the respondent to shift 
hts frame of reference 

R Oh, I, uh, I take care of things like lime 
off and, uh, figuring out, uh, uh, when the 
men can go on vacation without busting up 


the work schedule, and, uh, if, uh, a man 
has been around for a while and is about 
ready for a pay raise, uh, higher pay rate on 
the job he’s doing, it s up to me to recom 
mend em, sometimes if they want to pro 
mote a man to a higher job, uh, that’s on 
tny recommendation 


7 / Uh hum, uh-*How long have you been 
on this job? 

Oh a coupla years 

8 I You ve been on this job a couple of years’ 

8 This restatement of the response bungs 
additional specifictty from the respondent 
R Well, not quite so long— uh~lets see, I 
came on this job after Joe left, and that was 
a year ago Christmas time— its about a year 
and a half, really 

9 I Year and a half, I see “What were you 
doing before that? 

R I was laying track 

10 I That was for the same company, you 
mean? 

10 Part of the objective of the question in 9 
IS to determine whether the person’s prior job 
was m the same company Here the interviewer 
uses a direct question to ascertain tins infoimn 
tion 

R Yup, right here 
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fits altitudes on the foreman's role He broke into 
the discussion and returned to a topic which the 
respondent had mentioned earlier This was un 
fortunate^ since he lost material which might 
have been very relevant on working with the 
men 

R Well, what I mean is working with 
people— now, uh, I remember some of the 
things that used to seem good or bad to me 
when I was on the track line, and this way 
being foreman, I get a chance to try to make 
the set up a little better for the rest of the 
guys 

19 I You have a chance to help the men some 

19 This IS a statement rather than a ques 
lion It serves to summarize responses and en 
courage further responses 

■R* Yeah, I remember how it was when I was 
on the line, and I think I can make things 
better 

20 I Such as 

20 This probe is the same as ''What were 
you thinking of here^ ’ 

R ^Vell, like making it handy to get tools 
they need and arranging the work so they 
don’t have to work haid sometimes and loaf 
others things like that 

21/1 see Well now, lets take the other side 
of the picture for a minute, uh— *What are 
some of the things that you don t like about 
your job? 

R Oh, I don’t reall) know what to say to 
that 

*^2 / Uh hum 

R I, uh, I don’t like to complain you 
know, they ve been pretty good to me here 
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on things you lihe about the job, but I was 
going to ask you, uh-*What are some of the 
things you like best about your job you have 
now? 

15 One of the problems of an interviewer 
ts how {o fljfe a question when the respondent has 
given a partial answer under other questions 
This illustrates a technique for handling the 
problem The interviewer recognized that the 
respondent had mentioned something on the 
topic earlier, then he proceeded to ask the ques 
tion This avoids the implication that the inter^ 
viewer was not attending to the earlier discussion 
and serves to get new material 
R Well I think, uh, the thing 1 like best, 
like I was saying be{ore, is the higher pay, 
and, uh, uh, the security of getting a job in 
management 

16 / Uh hum— you mentioned higher pay and 
the security ot the job— are there any other 
kinds of things you think of there? 

16 This type of probe was discussed earlier 
—the summaty of coniersation, then the request 
for other responses 

R Well I guess you could say 1 like the 
supervisor s kind of work 

17 I Uh hum 

R It gives you a chance to work with the 
men and at the same time (interruption) 

18 I You say you like the supervisory kind of 
work Could you tell me a little bit more 
about what you have in niind there^ 

18 The interviewer felt that the respondent 
had not given enough information on the super 
vtsory work Hence the probe, which directed the 
respondent to this topic However, the inter 
viewer did not allow the respondent to exhaust 
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IS, uh, handicapping you in )our liork Uh 
what else do }ou think of in this respect^ 

26 Tilts was a poorly timed probe The 
respondent was talking about his supervisory 
problems at a level which would give real insight 
into his foreman’s role and its problems The 
tnleruiewer chose to focus the respondent on a 
new area rather than to follow up on the basic 
problems 

R Oh, I don’t think there s an) thing else 
I really ought to mention 

27 I I’m interested m what )ou ha%e in mind 
there 

27 Here again the respondent shows some 
resistance In this case the interviewer merely 
asks him to talk about these resistances In re 
sponse, the respondent mentions a problem area 
This was probably a more effective technique 
than making a more supportive remark 

7? Well, they have awful tight production 
schedules here 

28 2 xJh hum, and that affects you 

28 This follows up the previous comment 
Here the interviewer recognizes the attitudes tm 
phed 6y (Ae response It is a statement ruther 
than a question 

72 The, uh general foreman holds our sec 
Uon responsible for getting out a certain 
Amount of track each day It seems as if he 
doesn’t know anything but just 100 percent 
all the time 

I This causes you some problems, I gather 

29 This again recognizes the attitude un 
derlytng the remark Notice that the response is 
in terms of the attitude rather than content The 
real attitude comes out in the response after 
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23 I Sure, I understand that, uh, what we were 
thinking of, uh, that uh, on most jobs a per- 
son has, there are some things that he may 
not like as well as others, some things that 
he may actually dislike We are trying to get 
a general picture of, uh, what some of the 
things are that aren’t so good 

23 This probe follows the resistance which 
the respondent showed tn hts previous remark 
It js a general restructuring and is supportive m 
that It recognized the respondent's reluctance to 
be critical and attempts to make it acceptable for 
him to give negative statements 
R Well, one of the things you might say 
thats bad about this job is the condition we 
get the parts m 

24 I What, uh, what s that? 

Ji Well, what I mean is, these steel cleats 
that we have to put on the tracks, we bolt 
them on, and there's another section further 
up [he line where they're supposed to drill 
holes in the right places for us to slip the 
bolts in and half the time they do such a 
sloppy job there that when we try to put the 
bolts in place, we find they don’t fit and we 
have to spend time reaming out the hole, 
and when we do that, we slow down on pro 
duction and then the general foreman comes 
around and chews me out 

25 / Uhhum 

R And, uh, it seems to me that if they got 
things tied together better, that wouldn t 
happen We don t have a chance to talk that 
over 

26/1 see, uh, you’ve, uh, youve mentioned 
one thing, that the, uh, way you get the parts 
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change the question but merely defines what u 
meant 

R Yeah, we have em 

I Well, how, uh, *How do you get along 
with the shop steward^ 

R Oh, pretty good I guess I don t quite 
know t\hat you mean get along ' 

I Oh, I suppose what we lia\e in mind 
there is that we are interested in finding out 
how people perform tliese jobs, and how 
people in union shops who do these jobs get 
along, when they have to work together like 
this What are your ideas on that? 

35 It appears that the intewiewer was 
caught off balance by the respondent’s question 
He was not sure whether the respondent was re 
sibling 01 merely unclear as to what was wanted 
He responded in leims of the question objectives 

R Well, most oi the time, we don i have 
any trouble with each other I try to take 
care of my job and he takes care of his Of 
course, sometimes there are differences that 
have to be settled 

I How do you handle differences when tliey 
come out? 

36 This IS a direct question which is in line 
with the objective of the questionnaire item 

R AVell, like suppose a man figures his job s 
limed too tight He can mention it to me 
directly if he wants or he can take it to the 
shop steward Now, if the steward gets it, he 
can come around and well just talk it over, 
or if he wants to be nasty ibout it, he can 
file It as a formal grie\ance 

^ 1 see How do you usually work out these 
cases in )our section? 
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Question 30 Those leaders familiar with the 
principle of client centered therapy will recog 
ntze the technique This is the first place in the 
interview where there is real emotional content 
to the responses The recognition of that emo 
tional content helps to bring the interview to a 
level of discussing those attitudes rather than 
conversation at the symptom level 
R You re darned right it does— especially, 
like I was saying when the parts we get 
aren t quite right 


30 I Yes, I see 

R It seems to me that people higher up m 
management ought to find out more about 
how things are for usl 


^ You (eel it would be an easier job for you 
if people higher up knew more about vour 
job? ^ 

31 Again this brief summary helps the gen 
eral permissive atmosphere 
R Yeah, they don t come around to find out 
how things are really going-ii s just 100 per 
cent production, or else! 


32 


S3 


/ Yes, I see Well, let s turn to something 
else a mmute-uh. ‘Do you have a shop 
steward in your section? 


R 

I 


32 By asking this question at this time, the 
interviewer closes off a fruitful area of attitudes 
It IS interesting to note that at this point the 
respondent shows negative reactions He fails to 
understand the interviewer's question and quib 
bles over words Tim may well represent his 
resentment at being closed off 

You mean, uh union? 

Yes, I mean a union shop steward 

33 This clarifies the question It does not 
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general information about ^ou, for example, 

5 our age--ho\s old are )ou^ 

40 Having met resistance, the interviewer 
goes into more detail tn describing reasons for 
collecting personal data It is common in the 
research interview to get some resistance on these 
questions because of the concern over personal 
identification Interviewers usually use the tech 
ntque illustrated here They male a short state 
ment and ask the first question If this brings 
resistance, they give a more complete statement 
of purpose 
R Well, I m 33 

41 I ‘How man} grades of school did }ou 
finish ^ 

R ^Vell, I ne\er got a chance for much 
school 

42 1 Uh hum, about hoiv far did }Ou get^' 

R Eighth grade 

43 I Eighth grade 

R Had to go to ^\ork 

44/1 see Noiv that last one~*About ivhat 
^sould )our total income be this )ear for 
yoursdl and j aar smcuedtate fatnilj ^ 

R I don’t see ^\hat that s got to do iMth it. 

45 / Well, this IS another one of these, uh 
Items I rvas mentioning to you people i\ith 
different pay rates and salaries maj \er} r\ell 
feel different!) on some or a lot of these 
questions ue ask— for example, )OU remem 
her our discussion about bow )Ou felt about 
>our pa) a while back IVell it ma\ be reiy 
"ell that a person getting one amount of pa) 

"ould feel xciy different!) from a person 
getting a different amount This giies us a 
chance to make a statistical kind of anal)sis 
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R Oh, he usually comes around and tells me 
what he thinks, and if we can work it out 
together, uh, we don t make a grievance out 
of it-grievances are just tough for the union 
and tough ioT us 

SB I Uh hum. then m most cases you are able 
to work this out between yourselves 
R Yes, he’s reasonable Hes a Uule suit 
necked sometimes on the time study things 
Heck, I don’t do the time study either I’m 
in the same box he is 

39 I Yes, I see Well this covers about all the 
questions I wanted to ask you there There 
are, there is just a little information I would 
like to get from each person we interview— 
uh, "About how old are you? 

39 At tins point the interviewer ts ready for 
the personal data information He restructures be 
fore asking the personal data 
R 1 thought that, uh, these were gonna be 
anonymous and nobody was gonna care who 
gave the information 

40 I Yes, that’s right let me tell you a little 
bit about what we get here As I mentioned 
earlier, before ue started the intenieu, uh, 
we don t take people's names we aren t in 
lerested in identifying them at all We do, 
however, want to know something about the 
people we talk to because, you see the older 
people who have been in the company longer 
may feel differently from the people who 
ha\e been here a short time, and the younger 
people may feel differently from ihose who 
are somewhat older, things of that sort ^Ve 
are not interested in identifying you m any 
way And so, 1 ha\ e just a few questions of 
this sort Would you just give me a little 
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itiR seems appirent and simple-tlnt ts hem to isk questions ot 
people and get .nCormation from them ^Vhat ts lacking ts an under 
standing of the characteristics of a good intcivieiv-that is, the re 
quirements that need to be met before an inteniew can be con 
sidered good What are the principles of staiidaidization, of vahdi y 

etc Inch we are trying to acliie\c? u . 

Describing the goal for the interMCWcr cm be done partl> by 
helping him to understand the total research process and the part 
which he has in that process The inters lewcr needs to know how 
a study is designed, the general principles of samp mg an 
the data are to be analyzed This information will serve as a basis 
for his understanding of the interview m lelation to t ie o a 
research process These research principles establish the basis fo 
the interviewer’s job If tins orientation is successful, the ^ 
Viewer now sees what he is trying to accomplish throng is train 
mg Furthermore, this knowledge provides him with a basic under 
Standing so that he sees why he is trained in certain tec iniques 

The second training aim is to motivate the inters lewer We have 
implied m the discussion of ’goals’ that the interviewer has some 
real reason tor wanting to reach this goal It is unwise, ^ 

to assume that because the goal has been pointed out, he is highly 
motivated to achieve it The interviewer must feel that what he 
doing IS important and sigmficmt-he must have an enthusiasm 
for his work Although this is the usual part of a training functio , 
It IS well to point out that it is important to stress motivational 
aspects, for example, pointing out to the new interviewer why the 
study which he IS about to undertake is important, what its function 
IS how It will be used, why it is necessary that the data be collecte 
accurately, and things of that sort Another motivational factor 
which IS common in interviewers is craftsmanship-that is satislac 
tion with an interview well done, particularly if the situation has 
been a difficult one Earlier we talked about motivating the respond 
ent to communicate It is clear that it is difficult for the intervie » 
lo motivate the respondent if he himself is not motivated 

The third aspect is training m interviewing skills, or S 

to the interviewer the specific methods and tec niques v 
taake him an adequate interviewer It is the feeling o 
that in many interviewer training programs too muc i ^ 
given in terms of ’ rules and speafics-that is m terms 
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45 Again a statement of purpose was nec 
essary It is at this point that the new interviewer 
IS likely to get on the defensive and have the 
respondent refuse to give an answer A calm state 
meat of purpose usually overcomes this This 
respondent showed more resistance on these items 
than IS usual The reason may be that he gave 
considerable critical information during the in 
terview (Much of this does not show in this 
e'<cerpt ) He would be concerned about the pos 
sibility of having his responses reported to the 
company and identified with him 
R \Vell, I get seventy three dollars a week 

4G 7 Seventy three dollars a week 1 see 


SOME PRINCIPLES OF INTERVIEWER TRAINING 

cl..a hVl P“"> ‘he collection of 

n nlVob irh = •'■Sh'y complicated tecli 

Tthe techtf »£ ‘he interviewer It is clear that, 

need cl etr,r 'fccttve. interviewers 

deptds!“ , V ?, of the obtained data 

in uir^denend r* '«hniques are applied, which 

ThisTecil ■"‘-''--r IS trained 

have been foimd LT' '=?'bTt' 

major emphases The first is to rl^rT^ “‘"‘"S PtogTam has three 
Mcwinir In manv iMin trainee the goal of inter 

oI training is cleVr andThriJ’®”"'' ‘hilh the goal 

concerned with this aspect IMoTraa""? ho Btcatly 

to operate a typewriter elficiently 

task involves rapid and accurate r^, "T “ ‘ ‘h‘“ h“ 
Tor the lathe operator, the same eoal ‘'“’‘“1'°" 'yPe™'" 

interviewer is not so apparent mL ’n“ Tl""‘ 
ence in interviewing, wLther in tlie’^r*' 1“'’'' *"“* ‘"’‘P™ 

everyday lives we ofin ask people oues^ns ““H 

one sort or another To the new inte’rvfe I ® information of 
o me new interviewer, then, the goal of tram 
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the interviewer used techniques which were irritating or embarrass- 
ing. By analyzing his own reactions to being interviewed as he 
experiences the effects of the interviewer’s techniques, he can become 
sensitized to the reactions of respondents. The trainees who are 
observing have a chance to see the performance and eliminate errors 
in their own interview techniques. Barron (4) makes this statement 
about the use of role playing as a device for training interviewers; 

The use of role-playing or reality practice is being increas- 
ingly recognized as an effective means of translating the prin- 
ciples into methods, of learning the "how,” of getting the feel 
of doing something in a situation where one is not playing for 
keeps. In training which is directed toward improving skill in 
Interpersonal relations, it is offered as an effective way of bridging 
the gap between the formal study of principles, methods and 
techniques on the verbal level .and actual work with these 
methods and techniques. It offers an opportunity for practice in 
the kind of work like interviewing, where close supervision and 
training on the job are very important. 

In addition to the use of role playing as a device for trans- 
nutting skills, the use of phonograph recordings which illustrate 
various aspects of the interview and typical examples of interviews 
are very useful. They serve to point out to the interviewer what an 
actual interview sounds like and how an experienced interviewer 
handles a specific situation. They serve, too, as a basis for general 
discussion of interviewing methods. 

No matter how effective the training, it is unrealistic to expect 
die original training to make finished interviewers or that it will 
be equally effective with ail interviewers. One of the essentials of 
graining is that further training be conducted periodically as the 
interviewers proceed in their work. As the interviewers grow more 
proficient, they become more interested and more involved in the 
iic details of the interviewing process. They want to discuss specific 
^ypes of probes, the motivation of difficult respondents, etc. In such 
sessions, role playing is a valuable technique. It permits an inter- 
'"iewer who has a problem to act the role of the respondent and 
'bus portray the difficulties which he is having. The difficulties 

Usually be ironed out through the role-playing session, which 
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first thing you should do is this The second thing you should do 
IS that/ ‘ When )ou get a question of this sort, it should be handled 
this ^vay, etc The interviewer is presented with a long list of 
specific techniques which he is to use, but the specifics fail to add 
up to a general, integrated system or a conceptualization of inter 
\iewing This conceptualization can usually be developed in dis 
cussions of the research process and the role that interviewing plays 
in the research process, by showing how the other phases of the 
research process are dependent on the interview, and by demon 
strating how failure to follow these principles leads to error or 
invalid results Once this has been established, training on the 
specifics IS in order Methods of skill training have been summarized 
by Bavelas (3) in the following statement What appears to be the 
most effective method of training skills is a common sense one— 
watc 1 others, let us watch you, discuss and evaluate differences, and 
again This implies the use of informal group discussion tech 
mques an practice rather than lectures or dependence upon written 


experience is to have the interviewer conduct 
This, however, has the disadvantage that the 
the imon ^ ^ second hand report of what took place during 

‘hat t.nte The .deal 

presence of the ? he conducted in the 

fecTurne , /Tic!? and for all to discuss One 

that o? role f ^nd has other advantages is 

senels a T has been adap.rf to 

runs invoMng .n.Tr;eri:r;er.fo::h.pf 

ol ihe respondent '’idSv™/h'"'*’i/ ®^°‘‘P P'^^' P”' 

and responding to the intervfe!^'""" P”®°" h^ hnows 

P'a).ng"Ana./or per/oi " 

of the CTouD art nc ^ of ^be interviewer The rest 

■here i, general discuss.™'/t Ae'^hm P}“>'"’S 

used Many times Ihe trainee ee, '“mques that the interviewer 

of the respondent a, he docs Lt' of m'"'’ 

iiciscr By playing the role of Ihe ' P '^'^g rple of the inter 

perceive wirero'.ffe inln.e/ rt 'r”^ “'-P--''"'' 

tailed to get information and where 
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attention to rapport building 'probing and lecording of re 
spouses These techniques are also illustrated in the sample inter 
View and accompanying commentary 
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also provides the opportunity for the rest of the group to profit from 
his experience and to learn along with him 


SUMMARY 

The purpose of the present chapter has been to discuss the 
technique of the research interview, to offer a rationale or theoretical 
framework for the technique described, and to put the interview 
in perspective as one of the various devices for data collection which 
jie at the disposal of science 

We began with the postulate that scientific progress depends 
importantly upon the systematic collection of data and that this 
"nohes (1) a statement of specific research objectives, (2) definition 
0 the data required to meet such objectives, (3) determination of 
popu ation rom ivlucli these data can be obtained, (4) selection 
or deve opment of techniques adequate to evoke the data We have 
Wr 1“ ‘hat the interview can approximate 

adaoted *d ‘“'3rch, and that the interview is especially 

lehel. fee, “ “d perceptions 

bel.eh feelings, past exper.ences, and future tntent.ons 

of tuo Imr m '"ot.vat.on was discussed in terms 

It by”Xcr respondents perception 

goal or bring a^b "“erview he may help to achieve some 

(2) the direct erat^f winch he considers desirable, and 

M rpeVkmgi ® a "“r -P“"de"‘ realizes 

opinions and ideas and accepting of his 

instrument 'vh'ch^vould" s'^eTo TraTl T ‘treating an 

Without bias into tfrm^ j translate the research objectives 

"oiild, at the same time, TOist'^!!!'*^*'!'^ ‘“Pondent and 

in motivating the resnnn)i#.ra, retard the interviewer 

of queslionnlre co^cl.™ ':eraT™‘“" ^P^" 

guage, frame of reference, infoimalion le” ersZf ' '“d 

ing, and question sequence ^ ^ acceptance, vvord 

The specific techniques whidi the mk 
csole complete and frank responses werrrr"'"' T"" ''"'P''’’' 

e reviewed, with especial 
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attention to rapport-building, "probing," and lecording o£ re 
sponses. These teAniques are also illustrated in the sample inter 
View and accompanying commentary 
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CHAPTER NINE 


Observation of Group 
Behavior 


Roger W. Heyns and Alvin F. Zander 


Within the past few years there “a 

in the use oE observation methods m the s u y social 

These experiences have indicated that meaningful data in 

behavior can provide reliable and conceptually meaning 
Held studies as well as in “J^'"™has been accompanied 

This increase in the use of o . . ; .jon in observation 
by an increase in methodological sop „„mDle of certain 

methods. There is a growing “^^ervation schedules, 

typical problems in the developmen reliability. Many 

the training of observers, and to methodological 

of these problems have not yet been J ^ these areas which, 
research, but there is a good deal of ' investigator to 

until the necessary research is done, 

avoid some of the more common pitfalls. *„„e5 of observation 

This chapter will deal -We shall discuss the 

mstruments: category systems and ra ^ S ^ involved in their 

finished products of both types an ^ ^ ^ve shall begin 

development. To provide a focus for '...Ttirin 

by describing an observer team in an actua si 
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AN OBSERVER TEAM IN ACTION 
The Setting 

Let us suppose tint an obser\er team of tuo is studying methods 
of problem soh mg m groups Let us suppose, further, that this 
phase of tlie investigation involves the observation of a large num 
ber of groups in the field This means that the observations are 
being done m a relatively uncontrolled situation The team can 
control neither the kinds of variables which will be present nor the 
behavioral responses to them The specific duties which have been 
assigned to the members of this team as well as the way in which 
the) are to perform them have been dictated by the study design 
and the theoretical framework which have been developed 


The Entrance and Behavior of the Observeis 

The observers have been trained to make themselves as noi 
threatening as possible Thus they have made sure that the grotij) 
knows ahead of time that they will be present and tliat the group 
Js get^rall) aware of the observers purposes in attending the meet 
meeting place earl) introduce themselves 
e airrnan, and answer any additional questions he may have 
wnceming their purpose and the way m which they will function 
lovvard both the chairman and the gioup is positive 
undemanCng and supper, ne Both .he.r style \ndV intent 
the.r replies to any questions are intended lo make the croup 

mlerTh :‘tf ' T recorf objeaL^^Ict': 

fo^n^trflrm'^^d'a.'s ^ P" 

explains the cenmr™* begins one member o£ the observer team 

in the ir. , k k content of the discussion but rather 

suggests that the Z mo”np T""^ H'T 

no, mvolve them tn the observeni and 

nature of work Utej vv.ll be dotn- vv““the 

The observers take especa! p^nno^! ” SO-ng 

frnm intfla-fas- .k t ^ * precaution to keep their activities 

from .merfenng with the meeting They s.t as far away from the 
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group as space permits They avoid comersation with °ther 

Ld any ot^her communication which might reveal their attitude 
toward what is occurring in the meeting unless there is laughter 
applause, or some other reaction which they can appropriate y 
make without appearing to differ from the dominant climate in the 
room at the time 


The Data Obtamed by This Team 

Observer A codes the problemsolving process of g‘«“P 
observer B records the content of the meeting To test hyP°* 
eses developed for this study, the problem solving obsem' 
prepared category system ' He is responsible for co mg 
contribution of each member into one of a number of 
He also records which member made the oo"<nbut.on and to "ho™ 
it was addressed These categories are listed on a stan . ^5 

which facilitates rapid recording of these data T is o s 
the group interaction in terms of the categories ' , . 

below with their definitions By the time the o ^ 

he has memorized these definitions, of course, and hence 
not repeated on his observation form 


Problem solving Categories^ 

Goal settmg These conlribuliom have 
mg or suggesting goals or objectives ot 

tent They are concerned with ends t , , , ..hirh 

objectives goals or ends may be those of the -d-dual . h 
he IS trying to have the group atta.n they ">=> 
uients of accepted goals of the group or part 
Problem proposing These contributions serve ^ ' 

presenung a problem either to content or m proccdnre 

are concerned Mith means to ends or goT S of 

Information seeking These coniri u i j I or 

seeking to obtain information =" f of 

tcclinical nature The information sought 

1 At some point in ihe data collcclton 

^itnroduced to proNidc a rchabihl) check for jHusiranon are laketi 

-Tins set of catcgoncs and other examp . pyojeci Unbersny of 

the procedures uscti hy the Conference Researen j 

^hchlgan (8) 
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fact on tvhich the group decision is to be based or which has 
bearing on the decision Contributions seeking factual objec 
live, or technical information concerning the procedure of the 
group or an indnidual are classified here 
Information giving These contributions have the function of 
providing objective factual, or technical information either 
m the subject area or with respect to procedure The category 
includes the citing of examples or illustrations 
Solution proposing These contributions serve the function of 
indicating solutions to problems They are suggested means to 
ends Modifications of or additions to solution proposals pre 
viously offered are classified in this category if the context gives 
the contribution a solution proposing function 
^development seeking These contributions serve the function of 
attempting to obtain clarification of previous contributions 
They seek to determine what was intended by a previous con 
tribution what us implications are what inferences are per 
missible These frequently take the form of an inference stated 
as a question 


Also included here are contributions which facilitate the 
procedure of the group by asking the group as a whole or 
individuals to comment, indications to individuals that they 
have the floor, etc 


Dnihpmcrnpmng Comribul.ons here elaborate, make exnl.c.t 
enlarge on contr, buttons Included here are mferences from 
prevtous contrtbnttons self repet.t.ons or restatements by other, 
>yper of contrtbuttons 

To m da'r r" r ™thout tntent 

°h.s o ‘’Td“ Of ttopbed F.nally 

t oTsTr a « ’ T"*’""-'- »•>■* Pnrvtde the rat.onale, 

tton'm, rlsnra'nrr'rZayr 

tton interpretation, etc ResMn I"! ’ suggestion solu 

difficulties or objections are rcIudZhZ 

a^peeraent or app^alZlT, ZeZ''' ‘"‘‘'““"S 

.ncuded here are indication, S a™ ra.^rZ faT^t 
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another has contributed, whether approval o£ content ,s present 
or not This is a supporting comment in procedure 
summer, sec/iing ThL contributions ash, in effect, for a sum 
arv-e g , ' I’m lost, where are we now? 

Summery gmms These contributions summarize the groups 
progress to date They refer either to '“^stantive material du 
Ls?d over a period of time to conclusions 
group’s procedure over a period of time Summary statements 
of individual participations are not included here 
Non problem directing This category includes 
the tangential sort and a myriad of 
personal sort, such as ’ Give me the ash .ray 

opening a window? It includes statements which have no 
reference to the subject matter of the conference or 
group procedure 

While observer A notes problemsolving 

categories above, observer B subject matter 

content He keeps a running account of the ac j 

discussed in accordance with the following 

1 Notation of each topic discussed r„Uovr,nr 

2 Classification ol each topic ° ^ procedures or 

u Procedural a topic having to do witn p 

b rhrnuv:“lT;p“.c havmg lo do wilh the subject matter 

3 Noll' oTllture of the tash -nf-ntiug the ^oup a. 
each point on its agenda, using the following ca.egor.es 

a To am\e at a decision 
& To approve a decision already made 

c To recenc or gne information nn the 

1 Obscnalion of iihat actuall) happens to ea i 
agenda by nolu.g whclhcr the agenda item was 
<1 Completed 
b Postponed 

c Lett uncompleted trained to 

loll °n ’'observer’s .Mrmial " 

‘“'low carefully stated definitions in an Observt 
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When the meeting is finished, each observer is required to rate 
certain aspects of the meeting particularly the areas of coramunica 
tion, motivation, and interpersonal relations Each works inde 
pendenily, using the following scales 

1 Vnderstandttbdxty To what extent were the participants 
getting the meaning of one another 5 statements? 

0^2 3 4 5 6 7 8 9 10 

They were talking Communicated 

past one another, directly with 

there was much one another 

misunderstanding 

(Note This rating should include not only trouble with par 
ticular t^ords bm also more general conceptual processes 
such as difficulties in Ie\el of concreteness style of expression 


2 Opportumty to Communicate To what extent did the par 
ticipants have opportunity to talk? 

° ^ 3 4 S 6 7 8 9 10 

Never had Seldom had Usually had Had every 

opportunity opportunity opportunity opportunity 

to talk 

Note In some groups this car. be judged by the number of 
mes persons seemed eager to get the Boor but could not and 
he number of simultaneous part.cipat.ons In other groups 
*e paruepant, base already learned no. ,o try to talk be 
^00,1 •>' h f'w htembers hence, although 

to say 11, em hare little opportunity 

0123 


Nothing to 
gain or lose 


Much to gain 
or lose 
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4 Urgency How urgent did the group regard these problems^ 

0,2345 6 7 8 9 10 


No urgency 


Very pressing 


(Note This IS a judgment of the extent to ivhich the group 
felt It was necessary to arrive at a decision at this meeting) 

5 Importance How important to their organiration uere these 
problems regarded by the group^ 

vv o . A 7 8 9 10 


Of little 
consequence 


The very life of 

the organization 
depends upon it 

6 How formal uere the interrelationships among the people in 
the group? 

0 12 3 4 5 6 7 8 9 10 

Extremely Predominantly l-citgely Completely 

Extremely rr 

formal formol 

(Note Rate on bnsis of mode .nditiduals 

comments and number of asides ns d b, the 
indicating social distance among them ) 

7 Hou supportne and accepting was .he group of US members 

e A 7 8 9 10 

0 12 3 ^ ^ ^ — 


The group was 
permissive end 
highly receptive 


The group was 
highly critical 
and punishing 

8 Hou pleasaut was .he adeelue m.erpersona, a.moiphere of 

the group’ 

e d 7 8 9 _1 0 

Very pleasant, 
personable, worm, 
ond cnioyoble 


1 


Very unpleasant, 
quarrelsome, criticol 
and unfriendly 
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Below some of these scales there are instructions to raters con 
cemmg the cues which are important and the extent to which 
certain factors should be weighted in arriving at a final rating More 
detailed instructions of this sort for each item are again provided 
m an Observer s Manual ” 


Each observer also writes anecdotes describing incidents or 
conditions which he thinks might be important Specifically, he is 
asked to note factors which have a bearing on the observations 
obtained (their validity, interpretation), factors which should be 
included in subsequent observations, and the presence of conditions 
which might make the group unsuitable for inclusion in the final 
analysis of data 


This brief description of the steps a typical observer team might 
follow has been placed in a field setting The observation process 
in a laboratory is not very different, however The experimenter 
wou probably introduce the observers to the persons being ob 
swe an describe the activities of the observers More will be said 
about the introduction of observers and the problems of observer 
decorum in a later section of this chapter 

Now that we have described the activities of a hypothetical 
ready to turn to a discussion of the technical 
ures invo \ed in the nature and development of such a system 


CHARACTERISTICS OF AN 
SYSTEM 


OBSERVATIONAL 


descriDtion^nrtr concerned, first, with the definition and 

terns and nuingTcXs °Th?d systems category sys 

of finished nrnd„rf. t '^ascriptions present the characteristics 

■r. the development of the“e 


Category Systems 

One of the most useful devicpc ,1,^. i. 
nations ■„ quantuative form .s that oftS ‘‘r n?'"' 
separate categories For the purposes of 

staternenrdescr.h,„gng.venc,a^„fphenre:.r.row\mr!3ed 
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behavior may be coded, a category system consists o£ two or more 
categories A carefully developed category system provides a com 
mon frame of reference for observers and increases the likelihood 
that the relevant aspects of the total behavior will be note wit 
reliability 

The exact nature of the category system-i c , its characteristics 
with respect to the number of categories, the level of conceptua iza 
tion they involve, the applicability of the set to a wide variety o 
situations, etc —depends upon the purposes of the investigate an 
the theoretical framework within which he is working t oug 
It is possible to distinguish various types of category systems a ong 
a large number of dimensions, several dimensions seem to us espe 
cially useful for understanding the kinds which have been use 
the types of data which can be obtained by their use 
THE DIMENSION OF EXHAUSTIVENESS Some category 
developed in such a way that all the behavior observable can be 
classified into one of the categories m the set Ba es m er 
categories make up such a system (2) The 12 
have been developed with the objective that all verbal behavi 
small face to face group be codable into one or anot ® , 

A contrasting system is that used by Jack m her study o a 
and submission in a play situation (9) Her set of categori 
on forms of ascendant behavior, behavior not in these ^ 

'vas not coded In a sense, of course, all less than ex ^ 
systems are really exhaustive, since behavior . 

the categories is implicitly in a category la e e it 

This distinction is a real one and consists essen ally of 
determining how much of the total observable behavi 
'iMsified into a defined category to 

The question of whether or not a category sys 
exhaustive (in this sense) must be decided by the exp 
■'> ‘he light of hts purposes Two considerations seem ‘ ‘ « 
»>“ howeter The first is that much time can he ^ ‘U 


h ‘he instrument contains only those categories which are ey 

The r • .. . 1 .^ « 


- instrument contains only tnosc u-ivfax,.. helnvior not 

* ae second consideration concerns the nature .ir, d Es en 

“‘egorized when a less than cxhaustne category sys e 
hough no further discrimination is necessary than 

„ may nevertheless be important to know the 
of behavior in tins residual, undiftcrcntiatcd category 
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done only if one has a record of the amount of total behavior which 
^\as not specifically categorized 

THE DIMENSION OF INFERENCE Another Way of differentiating 
category systems involves the amount of inference that they require 
of observers Some category systems require that the observers coding 
the behavior proceed from the actual behavior which they noted 
to a deduction about this behavior At the other extreme are pro 
cedures which require the observers to place the behavior they saw 
or heard into categories with no requirement of inference Let us 
suppose that an experimenter is specifically interested in the effects 
of strict disciplinary practices m the classroom on the kinds of inter 
action of children on the playground At the simpler extreme, he 
might develop categories such as ‘Shoves other children,” ‘Calls 
other children names, ’ “Asks for help ” These behaviors are selected 
either because they represent the level of data the experimenter 
needs to test his hypothesis or because they permit later reclassifica 
tion into categories which are at the appropriate level On the other 
hand, he may use such categories as ‘ Shows hostility,” “Demands 
submission, ‘ Desires support,” Resents dependence ’ In the latter 
case the observers are looking at the same behavior but are making 
inferences concerning u The material which is coded is based upon 
these inferences 


When some inference is required to test the hypotheses, the 
essential difference between the two category systems is who makes 
the inference In either case, some theoretical system is required 
on t e asis of which inferences are made, either by the experimenter 
« hen he confronts the data presented to him by observers or by 
he observers when they observe the behavior In other words, in 
‘ ''''' incidence of behavior in cate 

nf ihp t ‘ ^ investigator makes the inferences during the analysis 
in make” ”’r the experimenter asks the observer 
inference! n'" ‘'«"'“cted him as to the kinds of 

descrihinn ih ^ '"ttruction actually consists in 

from oh “ t" how much inference to require 

memer Th^ '’"’n o' the expert 

menter There are, however, a nnmHn,. ^ j T l 

^ number of considerations which 

^re Ls?n h “'<= *Sre= of confidence 

one has in the clarity of the concepts being used and the behavior 
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vhich may be described in terms of these concepts For “ample, 

when we ash observers to make inferences concerning the motive 

or emotional state of the actor from behavioral acts, 

have empirical evidence that a number of phenotypica y ^ ^ 

behaviors have the same genotypical dimension or at ou 

about this situation is such that they may toe availab.litv 

of sophisticated observers is another factor w ic a ec 

ability of using category systems which involve ig eve 

If onL resources in this respect are limited, it seems »-h more 

desirable to define clearly the categories of behavior ° 

with a minimum of inference, leaving ihe inference task to 

“’’“ror mMHNSioNS some category ^V^-ms are toeloped 
within a single frame of reference, ihey require t e 
focus on behaviors which are, at one level of “"“P j 
homogeneous Others include categories whic 
havior along a number of dimensions This 
be applied to single categories within a system A" 
category system focusing on a single aspect ° of the 

developed by Heyns and Berkowitz (8) for ,jp,rribed at the 
problem solving process in decision. making classi 

beginning of this chapter) Each P"“°P'*"““he"pr°blcni solving 
lied into one of 12 categories on the basis o oroup 

function performed by that contribution Ot er are 

interaction, such as the emotional impact 

Ignored or only minimally represented ,ii,,s, ration ot a cate 

Bales’ (2) interaction category system is example 

gory system involving more than one „ tension 

Category 1. ’Shows solidarity, and '’a affecuve dimen 

release,’ seem to be descriptions of interac lo orientation, I 

sions Categories 5, 'Gives opinion b i^wes rj^'itv 

Gnes suggestion refer to the intellectual pro em 
“f the group „„mber ot dimensions 

There is, again, no pat answer to It should be pointed 

''hich should be attempted in a category s) j the number 

however, that the number of dimensions i a«are of and 
of frames of reference which the obsersers obser% 

'^ustuse A large number tends to reduce apeem j ^orc 
A further complication m the use of systems ln^olvl g 
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than one dimension is that the categories on the diflercnt dimensions 
may not be exclusive Thus, for example, at one level of analysis 
a question may be coded quite correctly as a request for informa 
tion At the very same time, it may be coded equally correctly as 
an expression of hostility This dilHculty can be overcome, of course, 
by permitting multiple coding This may not be desirable for other 
reasons, however Finally, when a single category includes behavior 
on more than one dimension, there should be strong theoretical 
or empirical support for the proposition that at another level of 
conceptualization these two dimensions are similar ^Vithout this, 
a category score would have no single meaning 


Discrete vs CouMjiuouj Calegortes 

Some category systems are so consirucied that the categories 
within them can be placed along a continuum Others, even though 
t ey have only one dimension, contain discrete categories, which 
cannot be located m relation to each other on any very clear con 
imuum Wt can illustrate the first t>pe b> referring again to the 
describing the playground behavior of 
/ , suppose that he has a number of categories into 

which agpessive behavior can be coded Category I. ' Mild verbal 
c , at^ry 2, Verbal threats and threatening gestures’ 
u physical attack According to one classification 

milH ^ ** ^ ^ continuous category system, ranging from 

itaUiame^ "" <>■> the adequacy o[ the theorel 

rore“fo J" r “ 

situations, however. .n®^rck™hnT-a!"’’“' 

concernmo- tho a When assumptions 

oped^adequately. they consu.ule a Cha^ U)""'" 

relatlon*!p^o oneZnorhcTlr™"” """t^h have no such 

solving category sjstem referred n7' ‘m '"'f'” 

such categories as supporting soluuon ^ 

developing on a single, clear^onr.nuum ’’”'"® '““"S =‘"‘' 
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Rnthiff Scales 

Simple rating scales are also often used to 
observations of a social situation. They may be used to <lescr be th 
behasior of individuals, the activities of an entire poup, the changes 
in the situation surrounding them, or many . 

Rating scales often provide more superncial and less ' 
than do well-developed category systems such as t ose jus 
However, practical limitations may force one to re y up 
method to cuide observations. 

By the phrase "simple scale” we mean a scale with a t^t of pomts 
which describe varying degrees of the dimension eing 
Observation rating scales have seldom been tohm'tted g 
"scaling" treatment in their development, probably hecau e it 
often difficult to get a sufficiently large number o tria 

one observation schedule, , 

HOW RATING SCALES ARE USED. Rating scaks ate most often 

used in either of two ways; (1) to record behavior at req 

vals throughout a sample of social interaction, or ( , f 

nature of fhe entire social even, after it has ended. An -aniP’'^;’] 

the former type was used by Lippitt and an er { v 

experiment on Scoutmaster training. Observers no e ..(-eived 

of Scoutmasters in Scout meetings before and a ter t ey [j j^al 

training. One five-point rating scale was used to r 

symptoms of group tension shown by the boys uring activity 

hy the trainees. The scale was marked vvhenever t n descrip- 

or the group atmosphere changed during the 

don of the scale and the observer instructions was as 

Phynraf symf lomr o, group tem.ou This -ale u concerned 

with the state of rennon evisung m .he group “ r - 
through physical symproms ft wiU 

ot psychological strain created in y ^ 

of a Scout meeting The tusump.ion is made n ^ 

IS physically tense to a noticeable ex en ^ ^ ^^ijellion 

psychological tension This psychologica goals 

or fear, or it may be tenseness caused y leaders, or the 

set before the group (by the °o,es to know if the 

hoys themselves) It will he difficult s 
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tension is goil directed inxiety or whether it is nn emotioinl 
reaction An anecdotal note on the source ol tension will be 
saluable if discernible This scale asks lor both sorts of tension 
on the hunch that most of the ratings will concern goal directed 
tension and those which do not will be apparent from the rest 
of the data According to physical signs how tense arc ilic bo)S 
at this time in the meeting? 

0 position— Can t rate 

1 position — relaxed The group is physically and psycho 

logically taking it easy This does not mean that 
they are in a state of rest or repose It means that 
they are carrying on the kinds of activity which occur 
at a Scout meeting without any apparent air of ten 
sion This may be a sprawled conversation group or 
It may be a relaxed game The boys seem comfortable 
they act and look the vvay jicoplc do after they rise 
from a good meal 

2 position— Re/axed Mark this category if tlie group is relaxed 

but not as greatly relaxed as in the category above 

3 position— ^fidd/eground This category is marked if the boys 

are acting as most people do most of the time The 
goal has a more positive valence than might be true 
in positions 1 and 2 There is a small amount of 
tension but it is not great enough to be expressed in 
physical signs of tension They may be physically 
active or sitting still Facial expression shows no 
apparent signs of strain 

4 ^ozmon-Reiiless This point is marked to describe group 

behaviors which indicate psychological tension Boys 
may be trying hard in a signaling contest or a 
written examination Sometimes tension may be ap 
parent m the purposeless movements by the boy 
(purposeless m the sense that they seem to have no 
relation to the group goal at the time) These are 
such movements as hand ur.nging foot twisting 
tongue chewing drumming and Other nervous 
mannensnis 

position Keyed up Here tension is very obvious Its physical 
signs are clenched fists or hunched bodies or extreme 
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signs ot rcsllcssncss or i tense anxious expression 
A football croKtl antcliiiif, th kick for a point after 
louclitloisn rsould be keyed up The keyed up 
belnsior may be sboun by boys rvlio are in any 
posture or state ot moseiiicnt During a fast basket 
ball game for example tb signs by winch it is 
recognized might depend on the pitch ind frequenq 
of shouts the facial gestures etc Hoys who are re 
tpiircd to sit Ihtoiigb 1 tongue lashing may be keyec 
up but tbc.r tension might be shown m facial 
gcblures Tiul inhibitcci ino\cmpnts 


Let us Ignore for the moment the adequacy of the 
on which this scale was based and the problem of re la i i y 
observation on i rough scale sucli as this it is clear t at i ns is 
attempt to get a measure of the significance of a large 
bodily movements which arc interpreted by the observers a 
tive of psychological tension and tint these many . 

tered throughout ilie gioup are gathered in a re ative y eco 
Has An eximple of a scale used to record behavior a. th end of 
a meeting svas described earlier in this chapter no i 
"as developed by Fouriezos Hull and Guetzkosv ( ) 

‘0 rate the amoLt of self oriented need behav.or shosvn by eac 
member ot the group This type of behavior is described by 
authors as follows 

[Self oriented need behavior] is not "ecessarily dneemd 
toward a group goal or the satisfactory solution of the F P 
problems U is direcled primarily to the ac - of the 
need itself regardless of the effect on the attainment of gr P 
goal 

The scale has 11 points ranging from 0 (no 
oriented need) to 10 (all behavior of the selforien 
follows 


^0 No expression of self oriented need 

, 1 Some slight indication ot selt-oncnle 


need behavior 


.2 

. 3 Some self oriented need behavior 


mdiested but nor 
prominent 
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.4 

.5 Considerable itUorienlcd need behavior 
_6 

_7 Almost all behavior of selforieiucd i)pe a grtii deal 

of expression 


9 

10 All behavior of the self oriented need type 


The ratings describe the amount of self oriented behavior each 
individual displayed during the meeting End-of meeting ratings 
such as these can be made on the basis of reflection and recall by 
the observer— the practice most typically followed However, Fourie 
20 S Huti, and Guetzkow asked their observers to tally instances 
of the expression of five different types of needs dependency, dem 
mance, aggression, status seeking, and catharsis Whenever any mem 
ber showed behavior which fell into one of these need areas, the 
observer recorded that fact Thus, at the conclusion of the meetings, 
he was able to make subjective, integrating appraisals of his tallies 
in the form of the single overall rating on the scale just described 
There is some evidence that such ratings arc empirically valid 
Use of the above mentioned scale m field studies has shown that 
groups which are high in expressed self oriented need tend to per 
ceive themselves as less unified than do low need groups (6), and 
persons of high prestige who do not have their prestige recognized 
^ over a series of meetings tend to increase in the amount 

of self oriented need behavior (14) 

Note, then that rating scales can also be used to summarize 
obtained by separate tallies on a category system 
s IS o ten necessary to create a pattern in the data which simple 
frequency counts on separate observation categories would not 
provide ® 


WHY ^TING SCALES ARE USED Ratings are used because they 
yield reliable and quantified data within time and personnel limita 
tions which often exist m studies usiug observers One may wish to 
have some measure of varied items such as the tnterest being shown 
by a group, or the amount of hostility, humor, or task directed 
behavior Assuming that one knew all of the cues which could yield 
an indication that one of these factors was present it would be 
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preferable to record the frequency of each cue as it 
could be a tremendous job, however, since there could be a large 
number of behavioral cues which might indicate the presence of o e 
of the factors listed above If one were interested in a large number 
of factors, the cues could mount to impossible num ers us, 
scales, which contain a variety of behaviors at eac 
scale, are more efficient since they can provide more ata per 
and more dimensions per unit of time The observer usi S ^ 

scale IS m the role of a human collating machine He 
number of acts throughout the group, integrates t em i ’ 

and makes a judgment as to which point on a num er o 
describes his interpretation of the varied behavior nrrur- 

Thus, at some points in a meeting when events “V ^ ^ 

ring so rapidly that it is impossible to record the " 
activities of each person, it may be easy to record a su 
inent along required dimensions by means of a ra mg 
m some cLs\ne needs measures of change m behavmr on^V “ 
the point where changes in other conditions occur j-scribed 
the rating scale on physical symptoms of ^f 

was used because it provided a quick snaps , from a 

affairs when the group changed program activi i (p 
teaching and learning condition to one in w t rapidly 
playing a game) Many new and important cues may pp j 

under such a ciangeJ state of affairs (perhaps a 
tension) so that one could not record them “ ^^^prd within 

on a leisurely tabulation of whatever It IS p rapidly 

an observers physical limitations since the can be 

change again before an adequate population o 
tabulatel In such a case, ratings provide useful 

Rating scales can be useful m the exp ^ P cn tjpe of 

a study If one is uncertain what the cues a 

l^ehavior, scales can be useful in defining t em ,vhich could 

interested m determining the nature of i le ^ expect that 
be used in Identifying supporting behavior Orm -gby-P^ ^„,cc. 
die content of the words said by a perso , re\cal clues 

^lie bodily postures, and his facial In this case, 

to the presence or absence of supports e e 
tequent r'alings, with no.at.ons of ‘^"’.o.nparc 

decision can be made by seaeral obsen 
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iheir ratings, and the cues they used, the)’ begin to be au*are of the 
behavior they use as indicative of supporting behavior. Further 
refinement of the scale, another period of joint obsers’ations, and 
comparative discussion make increasingly clear the clues that are 
most indicative of this behavior. 

Under ideal circumstances rating scales should provide data 
%v’hich are strictly comparable to diose obtained by the use of a 
category system. This ideal is diffiailt to reach, however, since the 
variety of behaviors which one must include within any one rating 
may create problems of reliability. Then, too, a rating scale which 
is discriminating enough for one type of behavior may be too gross 
for another. If the scale is so gross that most of the data fall at one 
end of the scale, the user rum into problems in the statistical treat- 
ment and interpretation of the findings. Such problems as these are 
treated in more detail in Chapters 6 and 11, on the construction 
of measuring devices. 

What criteria can one use to decide whether one should use 
single categories or rating scales? In general, whether one uses 
frequency counts or ratings depends upon one’s resources and the 
demands of the problem. The greater the precision required, the less 
one is likely to use rating scales. 


PROCEDURES IN DEVELOPING A SYSTEM 
OF OBSERVATION 


Syst^atic observation of social behavior has a relatively short 
ustory. ur information is still loo limited to give the researcher 
many rules of thumb which he can follow in the construction of his 
obsen-ational tools. ^Ve can, however, point out the major decisions 
whicli the experimenter must make, discuss some of the considem- 
tions which must be taken into account, and suggest some criteria 
which might be used in es-ahiating xW.s alternatives. In fact, it is 
clear to those tvho have had considerable experience in this area that 
there is no single solution to each of the problems; die best solution 
is in terms of the objectives of the study 

The importance of theotj- in the des'ign of experiments is often 
neglected. fVe need only pmnt ont here that the Uteoretical frame- 
trork plays a central role in determining the decisions which the 



Observation of Group Behovior 


■399 


experimenter must make m developing his f ’‘>§5' “ 
pamcularly true with respect to selection of the behavior to be 
lerved or rated and the definition o£ the categories 
these behaviors are to be placed Without a knovdedge o the pur^ 
pose o£ the experiment or research and the theoret g 

which the experiment takes place, no one can 
experimenter the dimensions with which he ought to ^ “n«rned 
or the amount o£ inlerence he ought to require o£ his obse^en -The 
experimental design and the specific hypol eses o neces 

dictate the level o£ reliability o£ observed f “ "kich w 1 b^nece^ 
sary Many other decision implications of the t core discus 

could be stated, several o£ these will be made explicit in the discus 

Sion of decision areas ^vhich follows orpcsine 

Some o£ the decision areas to be discussed are P™ 

lor rating scales than for category systems or vice * to^warrant 
are, however, sufficiently common to both techniques to 
their discussion in connection with both 


T'he name of Reference 

The frame of reference is involved, m bserwrs”aTd, 

m connection with the level of inference M'e 

m part, in the problems discussed under 

can explicitly restate the problem in ,he dimension 

>nierests Of rehabilit), observers must be Clear 

to be observed and about the vantage ^ be in 

observing, recording, coding, or "“"S “ , .nteraction along the 
siructed, for example, to observe the hich the mem 

thmension of interpersonal affect— i e , the exten dimen 

>>ers of the group are personally fond arc 

n IS fairly clear, the question arises pirticipants or 

react to the observed behavior as i > . inferences 

Vvhether thev are to identify svith tlie actor, ^ Man> 

about 
others 


Sion 

to 


nsion of interpersonal affect-i e , the ^ die dimen 

Of the group Sre personally observers are 

IS fairly clear, the question arises pirticipants or 

to react to the observed behavior as i > . inferences 

'whether they are to identify svith die actor, Man) 

®hout his motives These are different rame , reference 

tethers are possible Three of the more conimo 
problems will be discussed die recurrent 

■rnE CONTEXT AS A TRAMF OF REFER^^CF ibCOrCtlCallv 

tourccs of rater disagreement and one w ne t die ict or 

important is the extent to winch the socia c 
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behavioral unit is to be taken into account m the coding or rating 
Few would seriously argue that the context {the situation in which 
the behavior being coded or rated takes place) should be ignored 
It IS rather a problem of what part of the context, how much of it, 
IS to be taken into account In the coding of the problem solving 
functions for example, a specific contribution might serve to enlarge 
on or to develop a previous contribution This earlier contribution 
might have been opposing a still earlier contribution One coder 
might classify the present contribution as developing, whereas a 
second coder, having the larger context in mind, would classify it as 
opposing The experience of some workers in this area indicates 
that, if theoretical considerations do not dictate the answer, using 
the most immediate situational context as the frame of reference 
makes for the greatest amount of agreement This is especially true 
if interaction is rapid 


THE INTENT AS A FRAME OF REFERENCE Another prominent 
source of interobserver disagreement is the extent to which observers 
permit their judgment of the intent of the actor to color their ratings 
or categorization of his behavior This is often a problem when it is 
the explicit purpose of the experimenter to ignore the objectives 
implied in a given statement An illustration of this difficulty is that 
of the group member who is asking questions concerning the imph 
cations of a proposed solution to a problem The observer oriented 
m ignore intent svould classify this behavior as information seeking 
e o server oriented toward taking intent into account might code 
this same questioning behavior as attacking or opposing, basing 
his judgment on the manner m svhich the questions were asked 
the ,1. “f however, be interested precisely in 

iinr 1 K^t ° yhen this is the case two sources oE observer 

unre iabiltty may occur (1) observers may disagree concern.ng the 

oh ■" the tntenfoE the actor or (2) 

mto rf »■' degree to which they take .nten! 

tnto coustderatiou In short explicit instructions should be given 
ob eners ctnicerning the cues to use tor tdenttEytng intent and the 
extent to which judgments concprnin» ^ ° 

. L ° , '■^‘•cerning intent or motivation for an 

act should enter into coding or rating 

THE EFFrCr AS A FRAME OF REFERENCE A Specific aspCCt of the 
context problem ment.oned above is the .nfiuence ot the effect ot the 
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act on the coding or rating A statement in a group discussion 

might be coded by one observer, ignoring effect as a 

infomiation. It may happen, however, that the following r m k 

does not give the requested information It is a e ensiv 

on the basl of which another observer judges that the ongma ques 

lion was a critical or opposing act Disagreement beu “ 

coders frequently results from a differentia ten ency 

effects of acts. oJ perceived effects of acts, 

ments Whether or not this is to be permitted epen s 

and the interests of the experimenter It is clear t at t is 

tant decision area, certainly from a reliability standpoint and prob 

ably from a theoretical standpoint as well 


Size of Unit 

After the experimenter has ‘J'gj.^fX‘’p°oblem o! 

which he wants ratings or categorization, he P 

determining the size of the unit of behavior to be ra ed^or^ 
gonzed Failure to state clearly to the o servers 
of the unit of observation is a frequent soutt ability of dead 

tehabihty of the coding depends in part o 

ing what constitutes a unit of behavior sinele 

Roughly speaking, the size of ^‘,"pan.cipant-to a total 
act-for example, the contribution of a P gjudies of the 

period of interaction The observers m . . j^uacy of the 
Conference Research Project (8) made ratings period as 

problemsolving process and Time sampling 

the unit for some ratings, and single ac s randomly over the 

(categorizing or rating units of time distri obviously 

total action period) is an intermediate techniq 
■nclude a single act or all acts oot indc 

In categorization, the selection of the siz ^ fairly com 

Pondent of the category system itself This ..any act 

"ton practice of defining units in some sue i consider this 

which 15 classifiable into a single category reliability Thus, 

’rpect m the definition of units will perforce a ^ r,[ members 

‘ti practice, experimenters dealing with ver ^ .mnnssible to deal 

o' problemsolving groups have often found it imposs. 
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^\lth a total simple participation as the unit \ lengthy verbal state 
ment too frequently contains elements which permit classification 
in se\eral categories 


Sampling Methods in Observation 

It often happens that one cannot, or does not want to lecord 
all of the behavior that takes place in a given situation This may 
occur because the meeting lasts too long to observe continuously, 
too many persons are present interactions come too rapidly to re 
cord, or for any of a variety of reasons In such a case we are forced 
to develop a method of obtaining a representative sample of the 
behavior being observed This can be done in a number of ways (1) 
Attention can be concentrated on the behavior of a few of the 
members, ignoring the others present (2) Attention can be directed 
to each person, or to a number of persons in the group, each for 
a given length of time (3) The whole observation instrument can 
be divided into parts and the social setting can be observed in terms 
of each pan of the observation schedule for a standardized length 
of time (4) Observations can be made only when certain key be 
haviors have been introduced into the meeting (5) The most fre 
quently used system for obtaining representative samples of the 
behavior being observed is the time sampling system in which a 
standardized time unit is selected during which observation takes 
place The assumption is that these parts will be an adequate descrip 
tion of all of the events 


In general, it is risky to use sampling procedures unless one 
has an adequate theory to guide the selection of the sample As an 
Illustration, let us assume that we wish to observe a number of 
e aviors in a group but for some reason it is necessary to use a 
time samp mg procedure We make the decision to observe m terms 
of a certain set of categories for five minutes, skip ten minutes (dur 
mg which we may be observing with a different set of "spectacles’ ) 
and return for another five minutes with the original categories 
and so on Any social situation, however, is a changing set of activi 
ties, and we may discoser that certain events occur during the ten 
minutes we are not observing which distort the representativeness 
of the records made during our time samples Thus, we would 
rightly be suspicious of the adequacy of our data ^Ve could have 
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avoided this difficulty if ue had had some nonon of the range of 
possibilities of crucial changes in this group uhich could 
adequacy of our time samples and had ar'-ange * P 

procure in such a way that it coveted all of the crucial differences 
vhich may have distorted the representativeness of our time samp 
Only an adequate theory can help us 
changes are likely to be and whether they are likely 
difference to the coverage of the sample If we „ hoiii 

example, in the frequency with which group mem e 
we have certain personality measures perform in a ' makes 
mg to certain predicted patterns, we might 
any difference If parts of the meeting are not included 
It our theorizing leads us to suspect that certain 
procedure change the predicted relation betw«n about 

ures and performance in a group, we mus , 

time sampling It could happen that the stan ar 
we choose to observe may not be a ,^ou 1 d bt 

of procedures occurring in the group T 

biased by the particular time de the obsen i 

lime sampling only when we have a tl 17^^ miportaiit 

tions, since the theory can help us to .„„rded in the most 

parts of the data are bang buried or are being recorded 

edective fashion . , „„rK whether 

Sometimes it is possible to determine in a pi Confer 

lime samples of behavior are representative nroblem solving 

ence Research ( 8 ) found that the "-^^t nimumted mark 
behavior from one five minute period to „ould Iiave 

edly in real life groups Time sampling in 

been unwise „ hile using time sampling 

Another problem one may encounte behavior being 

that the psychological mcaningfulness _ ^nuos re^ctlons 

Obsened may be destroyed For example, rating fP ^ ,l,ai 

•o a given act of a leader may not give a 

It may be atomistic or incomplete 'C” Iciclcr's net before 

P^etc sequence of e\ents in a groups rcacuon o oihenMse 

according anything. howe\er, u ** f the croup s rcaciions 

tecord the behavior in a v\ay that includes aspect of the 

all of the implications of these actions thcorv of the siudv 

'°mext problem discussed earlier Here ngam the ibcory 
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helps one to make decisions U one simply needed to know the fre 
quency of oral statements which followed each act of the leader 
time sampling could be used with little fear that the data would 
be inaccurate If however one needed to know how constructive the 
behavior of the group was after a given act of the leader one would 
find that the i\hole sequence of events subsequent to the act would 
always need to be observed and that a time sampling procedure 
might cut off observation in the middle of the psychologically 
meaningful data needed 

To summarize there are certain criteria which will help us 
decide whether we should use time sampling in observation It 
should be used only when it is apparent that all of the behavior 
which is relevant to the study cannot be recorded The theory of the 
study can guide us in selecting the best sampling piocedure so that 
It does not destroy the reliability or the psychological meaningful 
ness of the material being noted 


TRAINING OBSERVERS 

The use of group observers means that we are using people as 
measuring instruments A good measuring instrument IS one which 
wi accurately measure at various times what it is supposed to 
measure If a person is to become an observing instrument he 
must be trained to see what it is required that he see This may be 
a simp e or a complex task depending upon the data needed It is 
an important phase of the preparation of a study using group ob 
servers an cannot receive too much ittention A v\ ell developed 
observation schedule will be only as good as the skill of the persons 

who are asked to use It ‘ 

in training observers are [airly obvious They are 
successful T'T* 'm practices which have been found 

serve:: are het*r'"''’ 
of ihLifotVa'^d^hlpTnosa S’, 

.» r ^ I L me study This is important since 

on for d tvrth a reason and thus a motiva 

1 ®l '''n ^ ""Ponant fiowever is tlte [act that it 

explains why tlie observation schedule is constructed as „ is Once 



Observofion of Group Behavior 405 

the observers are thoroughly familnr rsith the theory and purposes 
of a study, the trainer can communicate tilth neophyte obsjters 
concerning the inner ■\\orkings of the processes eing o 
boundary conditions surrounding certain categories an 

for perceiving certain familiar behavior in an unusual as lo 

2 It pofstble, tt IS helpful if the trainees attempt °bsert a“° ^ 
without the aid of a refined observation schedu e “ ™ , 
they would watch a situation comparable to one t ey 
the actual study and would attempt to identify as many ° 
vant behaviors as they could The discussion among t e o 
the events and behaviors they have seen accomp is es se\ . 
First, It makes the observers aware that behavior can e 

might not ordinarily be noticed unless they were oo ‘"o variables 

become sensitive to the fact that the behavior, or ° 
which they will he asked to describe, can be foun i I 

10 watch for them Secondly, this unskilled 
observer s part will reveal the need for operating i careful 

terms of a carefully defined set of categories e record 

agreement among them as to what to watch for an 
ihe behavior will become apparent as they ^1^^ signifi 

''^agreements among them as to what , 55 and so on 

eance of certain events as to what to call certain a jnstrument 

3 The observers are ready, now. for a more mstrume^^^ 

and the observation schedule may now be '"i™ ““ completed 
mg that the process of developing it has Y ^^ce the 

and that the schedule is prepared for the^ observation 

nbsetvers are always struck by the comp Y observation 

instrument, no matter how simple it may ^[^5 observer 

^^egories or rating scales will look more comp counts on 

*'1 first glance than it turns out to be ■" " °"be planned 

•ns proper reassurances and practice pr explained and 

Each of the items m the observation sc . , r obserscr 

'l^cstions are answered In most cases it xs e P , purpose 

" provided With an instruction booklet whic ^ j cues which 

° ^be study, the purpose of each item to e o . jj rnargiml 
be used for Lch category, the solution f"-- c.mnv 
'be nature of an adequate notation ^ I’ ,1,5 observer 
" procedural instructions, etc Obvious y, given phe 

“ ''B'lired ,0 make inferences or interpretations B 
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nomena the more specific and detailed must such instructions be 
to achieve clarity of understanding 

4 The observers are now ready for trial use of the form It is 
usually adequate to make the first attempts while they obser\e a 
group role playing the type of behavior they will be observing The 
\alue of such a procedure is that any trainee may stop the role 
playing whenever he has a question concerning the proper coding 
for a given event 

5 This try out is followed by extensive discussion of the experi 
ence The trainees will have had many problems in selecting proper 
categories, sampling keeping up with the events, etc These ques 
tions are answered by discussion and further practice It often hap 
pens that new observers have suggestions for revising the observation 
schedule which make considerable improvement in its efficiency 
as a measuring instrument 


6 Whenever the study allows, it is advisable that the observers 
ha\e an opportunity to make a pilot run on a group like those they 
will be asked to observe This will assure that uniform practices 
are de^ eloped for such problems as introducing the observer, eti 
quette of the observer while watching the group, and any additional 
problems which arise in the actual test situation and which were not 
foreseen in the role playing and schedule development phases 

7 Either the pilot run or a later trial will serve to provide data 
for determining whether the observers are doing a comparable job 
\ henever possible, it is best of course, that the research worker 
be assured that his team of observers are reliable before the actual 
data gathering begins 

In genwal, one can expect that the observers will have the 
greatest problems on those categories which require integration or 
collation of complex phenomena They will have the least difficulty, 
*^'nse events which are simple objective occurrences 
which require little insight or sensttmty on the part of the ob 
seners Studies of coder reliability (7) have found that there is most 
disagreement on data which are complicated and demand much 
ference Since an observer functions as a highly trained coder, it 
IS qune likely that he w.ll have simtlar problems 

This suggests that the skills required of an observer cannot be 
performed by all persons equally ,«n, , 1 ., 

training they may hate had One may find that a group of observers 
have comparible ability to understand the phenomena involved in 
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1 but that some ot them 
a study and to discuss them intelhg J group of interacting 
may not be able to “see these , some insights which are 

people. A study by Luszki (H) relation between sensi- 

relevant. She found that there is a p /empathic ability) an 

liviiy to the feelings and emotions o situation. Persons 

the ability to “see" what is able to see what is hap- 

t\ith good empathic ability (1) (2) have good persona 

pening in the role performance o used in the stu y, 

adjustment, as measured by ’ . similar to the evaluations 
(3) have insights into themselves w i ^gble. positive, and secure 
made of them by others. (4) ^^"vorable 

feelings about the self and favorable perception by others 

(5) have a more consistent an in persons will not do equa y 

One may assume, then, fpfeUpetence may force the 
'\ell as observers and some tes i © researc 

experimenter to retire less caP*’’)' observers often make valuable 
As tve have already stated, Thus u is im- 

proposals tor improving the o’’®® . to^rticipate m making sug 

porLt that they have the oPP°"i^p^g may be more effective it 
gestions. In soml cases, in fact, "’"^obstructing the obse™ 
*e observers participate m all pf observers. One pven 

schedule. BavLs (3) trained J^mion schedule m a manner 

haining in the use of a P^^P^^j^^^^rother group part.cipa ed in 
similar to that described earlier the beginning. In the 

the construction of the categor.« 3 P.dy the ^oup was 

fight ot its knowledge of t"® categories, tlje 

able to determine the t'=:"'®®/' !efas reports, in th.s unpublished 

each, and the rules for their use. in the development of 

study, that those observers more quickly (as "d hy 

the observation scheme were • ood reliability amo g them- 

the lengrh of time it ■<><>,'; ‘^..^^lio were told what d- “ ^tiei 

selves) than were those o ihoug 1 » Sto.,p 

meant and how they ch'cn to them. 

had more instructional tint b 


RELI 
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VALIDITY 
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unique problems which arise in connection with observational 
instruments 


Validity 


The problem of the validity of measures of social interaction 
har for a number of reasons often been slighted A brief discussion 
of these reasons may illuminate the problems which arise 

One way of phrasing the problem is to say that the validity of 
a measure varies with the degree of relation it has to an external 
independent criterion This, as Stouffer points out (13), is a problem 
of prediction As was indicated m Chapter 6, there are actually two 
kinds of prediction situations In the one case there is a generally 
accepted measure of the variable in question and the validity of 
the new measure being developed is determined by its relationship 
to this other, independent measure One of the reasons for neglect 
ing the validity of measures of social observation has been the lack 
of such external criterion measures It is difficult, for example, to 
think of an external criterion of a category such as ‘Shows de 
pendence 


The second kind of prediction situation might be described as 
the prediction of a relation which is derived from a well formulated 
theoretical system In this case, if the proposed measure makes dif 
ferential predictions concerning the behavior of people in a social 
situation and these predictions are confirmed in an experiment, 
one could argue that the validity of the measure had been demon 
strated Let us take a simple illustration Suppose that we had 
aval a e to us a valid measure of feelings of hostility toward 
aut only figures Suppose, further, that our theory led us to state 
* ^ itions under which hostile behavior in a social situation 
^ ^^J'ction of these feelings ^nd suppose that 
on t e ans of the lest scores, we organized two groups one com 
non hostile people, and that we 
o serve t cm in such a situation If, then our observer measure 
of amount-of hostility shown differentiated between the two 
groups, we could say that u was a valid measure of hostile behavior 
ometimes i e ind of prediction one makes is even less direct 
than in this illustration For example, one might predict that two 
groups of people differing ,n the amount of hostility shown in a 
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social situation might differ in their receptivity to influence attempts 
If, then, the measure of hostility shown predicted the amount of 
change which actually occurred, one might argue that the validity 
of the measure had been established The statement made earlier, 
that the theoretical formulation must be precise, ought to be under 
scored In other words, the conclusion concerning the validity of a 
measure in the last instance given depends on whether or not this 
relation might be predicted 

Sometimes experimenters are satisfied with the validity of an 
instrument when it relates in a significant way to some other van 
able This is a pure case of being satisfied with the validity of an 
instrument on the basis of its predictive efficiency— it is what Cron 
bach (4) refers to as empirical validity, in contrast to logical validity 
Logical validity asks ‘ Is this measure measuring what it is supposed 
to measure? 

One of the difficulties involved in establishing external criteria 
in social observation research is that social psychological theory 
often prevents ones havingfaith m what might superficially appear 
to be a satisfactory external criterion Suppose, for example, that 
observers were asked to note the extent to is'hich the group members 
were satisfied with the leader It might superficially appear that 
one way of checking on the validity of this observer rating would 
he to ask the members of the group ihcmselves how satisfied ihey 
Here with the behavior of the leader There are good reasons, how 
ever, for thinking that in many circumstances it would be impossible 
to accept the report of the members as a validity check There 
ttnght, for example, be strong social pressures against reporting 
dissatisfaction with the leader Thus in many social psychological 
experiments, the relation between obsener ratings and external 
criteria is a theoretical and not a methodological problem 

Most of the validity problems with social obseraation schedules 
ttrise in connection i\ith category or rating systems requiring a greit 
deal of inference on the part of observers There are many category 
systems which require little inference and the \alKlity is estab 
hshed by the content In these cases the validity question is not 
•mportant or critical For example, the problemsolving category 
system described earlier which has been utilized m describing the 
problem solving beliavior of a group has in it categories such as 
Proposes solutions ' 'Gives information' These categories hive 
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^\hat Guttman calls internal validity (IJ) In these category systems, 
the validity is established on the basis of acceptability There is 
common acceptance, for example, that a question is a question or 
that an item of information is an item of information Validity 
based on this common acceptance requires more than anything else 
a precise definition of the cues to be used in assigning a response 
to a category system 

Another reason for neglect of validity problems is the nature 
of the vanables with ivhich social psychologists are concerned They 
are frequently complex highly inferential vanables for which ade 
quale external criteria are rarely available The social psychologists 
with their social observation systems are thus often in much the 
same situation as are clinicians with their complex, highly infer 
ential diagnostic categories which are based on projective test results 

In spue of these admittedly sizable obstacles to the establish 
ment of the validity of an observational system it is clear that 
researchers m this area have not given the problem of validity as 
much consideration as it should have received Many of the tech 
niques mentioned in Chapter 6 for establishing the validity for a 
measuring instrument are appropriate for use m observational sys 
terns The reader is referred to these for precise designs which might 
meet his validity problems 


Reliability 

The reliability of observational instruments has been much 
more a mailer of concern to investigators of social behavior than 
has validity For purposes of exposition it may be useful to make 
a distinction that is not often made In assessing the reliability of a 
s)stem of social observation it is necessary to differentiate the relia 
bihiy of the behavior being observed from the reliability of the 
categorization or rating which is made of that behavior In other 
uords the reliability of the observer and of the behavior are sep 
arate problems It is clear, of course, from this distinction that the 
consistency of behasior is a subsUntive problem, whereas the con 
sistency of an observer is a methodological problem Once the 
consistency of an observer has been established, it becomes possible 
to tackle the problem of the consistency of the behavior 

The task of assessing degree of agreement among observers in 
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the case of a category system, breaks down mto at least two separate 
reliability problems The first is the extent of agreement among 
observers with respect to the number of units coded This is essen 
iially the extent to which the observers agree as to the boundaries 
of the unit which is to be cat^onzed Guetzkow (7) has pointed out, 
as has Bales (2), that unreliability in this area is an important factor 
affecting the reliability of categorization itself It seems clear that 
until there is rater agreement as to the boundaries of units to be 
categorized, there is little purpose in assessing rater agreement on 
categorizing The second principal task m assessing reliability is 
to determine the extent to which observers agree on the category or 
rating they assign to a specific unit of behavior 

The most frequently used statistic m appraising degree of 
agreement between observers has been the correlation coefficient 
This IS especially useful, provided the assumptions underlying its 
use can be met, because the extent of agreement that is being ob 
tamed can be evaluated in terms of a fixed standard It is a separate 
question, however, whether in a given case one needs a correlation 
of 0 7, 0 8, or 0 9 to be certain that one’s degree of agreement is 
satisfactory It is apparent that the question immediately becomes 
satisfactory for what? In other words, the experimenter or invest! 
gator must ask himself the purpose of his observational scores His 
theory will indicate the extent to which large or small differences 
are to be expected It is a truism that -where fine differentiation is 
necessary the scores must be more reliable than where gross differ 
ences can be expected It is impossible to state categorically that 
observational scores should be at sucli and such a level of reliability 
to be useful, for the usefulness of a score depends on the use 
to which It IS to be pul 

The second principal statistical device which has been utilized 
in characterizing amount of agreement between observers has been 
^ percent agreement score This is essentially a matter of computing 
the percentage of tlie total number of items winch were classified 
m the same way by the two observers or b) all the observers com 
bined Sometimes it is useful to modif) the equation slightly bj 
having as the numerator the percentage of items on which there 
"''s agreement, and in the denominator the sum of ilie items on 
''Inch there was agreement and disagreement It is necessary will) 
cither of these equations to have some fixed number of items sub 
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ivhat Guttraan calls internal validity (U) In these category systems, 
the validity is established on the basis of acceptability There is 
common acceptance, for example, that a question is a question or 
that an item of information is an item of information Validity 
based on this common acceptance requires more than anything else 
a precise definition of the cues to be used m assigning a response 
10 a category system 

Another reason for neglect of validity problems is the nature 
of the vanables v.iih which social psychologists are concerned They 
are frequently complex, highly inferential vanables for which ade 
quate external criteria are rarely available The social psychologists 
with their social observation systems are thus often m much the 
same situation as are clinicians with their complex, highly infer 
eniial diagnostic categories which are based on projective test results 

In spue of these admittedly suable obstacles to the establish 
mem of the validity of an observational system, it is clear that 
researchers in this area have not given the problem of validity as 
much consideration as it should have received Many of the tech 
niques mentioned in Chapter 6 for establishing the validity for a 
measuring instrument are appropnate for use m observational sys 
terns The reader is referred to these for precise designs which might 
meet his validity problems 


Reliability 

The reliability of observational instruments has been much 
more a matter of concern to investigators of social behavior than 
has sahdity For purposes of exposition it may be useful to make 
a distinction that is not often made In assessing the reliability of a 
system of social observation, it is necessary to differentiate the reha 
bihty of the behavior being observed from the reliability of the 
categorization or rating which is made of that behavior In other 
words, the reliability of the observer and of the behavior are sep 
aratc problems It is dear, of course, from this distinction that the 
consistency of behavior is a substantive problem, whereas the con 
sisienq of an observer is a methodological problem Once the 
consistency of an observer has been established, it becomes possible 
to tackle the problem of the constslency of the behavior 

The lask of assessing degree of agreement among observers in 
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determined for each set of conditions, and the degree of reliability 
desired by the investigator ivill then determine the number of 
judges he will need Using more judges to increase the reliability 
of observer scores does not seem to us to be a satisfactory substitute 
for improved precision in definition of the category or the trait 
to be rated, noi should it be used in heu of more precise elaboration 
of cues to be used by the observers in assigning behavior to cate 
gones, or instead of adequate training of observers 

In some cases the investigator may find it useful to combine 
categories to obtain a satisfactory level of rater agreement This 
IS true because often the lines of demarcation are fine and the dis 
tinction is difficult to maf^e Il'here it is possib/e, without introduc 
mg conceptual confusion, the investigator may expediently combine 
categories between which observers have difficulty discriminating 


THE RELATION OF THE OBSERVER TO THE 
GROUP BEING OBSERVED 

One of the most frequent questions addressed to the investi 
gator who reports a stud) which used observers is “Did the pres 
ence of the observers influence the behavior of the group’" Usually 
he must answer that he has no evidence that the observers influ 
enced the results m any way, but that they might have It is quite 
conceivable that the presence of an observer ma) be an important 
'ariable This depends upon the miure of the group, the type of 
observation, the nature of the groups activity, the nature of the 
variables being observed, and a number of other things Arseman (1) 
found that the simple presence of an adult sitting near the door 
seemed to lend assurance to a group of nurser) school children 
Yei the presence of observers was a threat lo young boys at a summer 
camp, according to Polansky (12) The influence of observers is a 
nicihodological problem which needs more careful study 

Deutsch (5) found that the members of small groups which met 
ftcijuemly over a period of three weeks were aware of the presence 
of observers at the beginning of their work together but Iiad become 
almost oblivious to them by the end of that period Half of these 
groups Vvere constructed in such a vvay that the members cooperated 
vviili one another lo achieve a common goal the rest of the groups 
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jected to classification Guetzkow (7) has provided statistical dencp 
for evaluating the level of agreement 

Bales (2) illustrates the use of the chi square to evaluate degrtt 
0 server agreement He points out that his use of chi square is 
more conventional applications since his w 
Thp a situation which does not represent random samplmj 

the advantage of chi square is that it does not require 

the as.ump„„„, „£ .he usual paraLlnc techniques 
concernfri”!* point out that the investigator ought to be 

analysis of th d ^ of the measure actually used in ibe 

Portance wheth« nhc!""* a matter of relative am 

of units of agree with the respect to the number 

with which assigned to a specific individual if the score 

m each cateeQrv*”''^J^*^*°^ concerned is the number of units 
reliability of ^ group as a whole In that case the 

group reliabilirv scores is not of concern as long as higb 

of the caieecrizlfi^ Present Another illustration is the reliability 
^^egory score for ” ^ opposed to the reliability of * 

observers as to ttidividual That is, the agreement beiwecij 
the categories m of responses categorized in each of 

observers show relai^ may be very high even though the 

®3ch Item This conri'^^ ^ agreement on the categorization o 
of t^tegonzation ar#» Occurs, of course, only when the errors 
investigator must ^ ^ The point is, however, that the 

then assess the reliah i what score he is going to use and must 
The section con * score 

of a category sv.f""'* Problems involved in the organua 
c made, many of emphasized certain decisions which mus 

' he immediately ure.!l "’"“''srations of reliability m ni>'’“ 
sewers also emoh?! ^^ction concerned with the training o 
'f^*»'>tyof,he.X""«'' problems with the ul..n-»' 

^“■gnivue. for "ew Detailed procedures, stal.st.cai and 

« observer ca.egor.r,™‘""'S reliability of an observer raimg 
“""stia However, |e ? 

With in'’’ ‘iemonn™”?' ''™ *’"'*• remarks 
the o ^ oumber of ,nd reliability of ratings increas 

'^daid depends .npa« “P?” 

■ration seems to hoL"'l""“ making the ratings, bul ‘ 

In any case, the reliability should W 



Observotion of Group Behavior 415 


served, the group members w.Il perceive them as such For most 
Situations that is probably best 


SUMMARY 

... 'VP“ 

..d »„.s K.l= A »«!.., •I““ ".S 

more categories or statements describing a c ass ^ ^ 

which observed behavior may be coded >7 V j 

the followmg dimensions Zt ln/ercl- 

observable behavior is classified into a ® required 

the amount of inference (concerning ^ J different 

from the observers Number of 

frames of reference required by con 

the extent to which the categon nTipstions as How 

tinuum There is no simple ansuCT to sue ff ^ j 

exhaustive should the ih^decisions in these cases 

require? The principal factor mlluenci g research The 

should be the theoretical frameivor S'* ® should be those 

second set of considerations affecting t e available 

concerned with the competence or '_^>°fJ“rrinB on 

observers, since decisions in these areas hate important bearing 

reliability and validity of observer scores u^invior of mdi 

Rating scales have been used to describe ^^havior ^f . 
viduals as well as the activities of an entire ^ P ,^(jr 

used to record behavior at frequent after 

action period, or to assess the nature o an 

It has Lded Rating sca^ are P-““'f/„;^,tUtauon of ill 
number of factors are under consi ,couId be an cnor 

occurrence of beha^ lor relevant to tliwe I ^ 

moustash They of clear definitions 

investigation as a device tor me i 
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were m a situation in which the members competed with one 
another It is interesting to note that the competitive groups were 
much more conscious of the obseners than were the cooperative 
groups Thus, procedural differences within a gioup may influence 
the way m which the observers are perceived 

Polansky (12) noted that changes took place in the perception 
by boy campers of the purpose and role of cruising observers Early 
in the summer, research observers were well accepted by the camp 
ers, who had been told that the strangers were neutral persons who 
were interested in learning how they could improve the camp By 
the third week, however, they had become the objects of aggression 
by many of the boys The observers decided that their role was too 
ambiguous and represented a threat to these boys, many of whom 
were rebellious against adult authority It was felt that the campers 
were projecting their feelings of hostility toward adults onto the 
o servers ter the observers changed their behavior to become 

friendly, they found that they were no 
longer rejected by the campers 

an cautious about geneiahzing from such 

the protm ^ ^ laboratory situation, for example, where 

doubt working toward a decision, it would no 

arrangement, T^y ‘th “ 

screen with the group aware rh^ oneway 

With the irroun wnr, f there, or behind the screen 

no d.Cference in the heWto’r oTt" 
uted to the influence of these vanm. 

Thus, It appears that e j ? of the observers 

social situation When in douK.'*’’ sometimes influence a 

of the observers to the aroim ’ '* “ '*’<! function 

necessary naiveti of the subiens"-^'”''^^’' 'P°‘' 

any special conditions such « those 

ing all of the external signs of a nitTe f I <>2), by show 

Itl-e a person who is haLg L wa“L°n 

6 reactions to the events being ob 
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of dimensions and the specification of the relevant cues Under ideal 
conditions, rating scales should provide data which are strictly com 
parable to those obtained with a category system with continuous 
categories 

In the construction of any observation system, the investigator 
must make decisions concerning the frame(s) of reference to be 
used by the observers This involves deciding (1) how much of the 
context of an act is to be taken into account in the coding or rating, 
(2) whether the judgments as to the intent of the actor are to affect 
the classification of the act, (3) whether the consequences of the 
act are to affect the observers scoring Failure to decide these issues 
IS a common source of lack of agreement among observers Another 
decision area has to do with the size of the unit to be categorized 
or rated Units may vary from single acts to total meetings Failure 
to specify clearly the unit to be assessed also affects interobserver 
agreement Finally, there is the problem of deciding whether to 
record all of the relevant behavior in a given interaction period 
or to sample the behavior In general, there is considerable risk 
involved in sampling unless one has adequate theory to guide the 
sample selection 

The use of observers means the use of people as measuring 
instruments This requires the careful calibration of personnel The 
training process requires that the observers (1) be familiar with the 
theoretical framework of the purposes of the investigation, (2) have 
experiences in which they become sensitive to the dimensions under 
consideration, (3) have extensive training experience with the pro 
posed observation schedule, including a trial run, preferably with 
a group like that which they will be observing 
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major importance for problems, 

in this section deal with -.patch is collected in what may 

Much of the data in swa ^ translating such data 

be called qualitative '“™- J nmre rigorous analysts are 
to a form which can be cpciion. 

presented in the first chapter o 1 1 nrevalent in almost any em- 

Problems of scale “nstruct.on -e p^^lcms 

pirical field. It seems, hosvever, • handle adequately 

which have proved adequate in o j jn social research. The 

most of the scaling problems encountered 
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One of the basic skills reqmred ‘^Ytenal remarkably 

“ore! r'- "and 

deifying ordering "“'’'‘"S’ groups of 


quantiCyingp 

CT 'symbolic'" pToducts of “"^ 17 Jrermh" which 

this chapter we shall consider some o t principles involved m 
tnay be analyzed systematically, the majo . data,” some 

converting symbolic “phenomena t> made m con 

criteria useful in guiding decisions t a practices found 

structing the system of categorization, ^ yj^bolic materials 

tobehplrrfTTi in t>iP artiial ofoccss of catcg ^ S recording 


AAA uaA:iui ill j some pi wk-Ai*-'-'' -- 

-ting the system of categorization, ^ yj^bolic materials 

e helpful in the actual process of categ ^nd recording 

The special problems associate wit i .^jjgncal manipulation 

symbolic behavior and those involved in tn rhis volume 

of the processed data are treated m ot e topics m separate 

Although it IS convenient to discuss t ese s decisions about 

chnnta^^. .. »e,Ti, fr* rpalize that, m pr * ^ , total 


^iinough it IS convenient to oractice, decisions ituuA.A 

‘Napiers, it is important to realize tha^ lu p ^ ^^jal 

analysis of these materials cannot e jjre data The 

plans tor the collection and statistical tma hmitations 

Ways in which the data are collected wil ^ d, in turn, the 

an the types ot analysis which will be feasible A 



second chapter in this section presents an approach to scaling which 
IS perhaps more adaptable to this field 

Testing the statistical significance of research findings has 
always been a problem where the findings are based on relatively 
few cases and the distributions are highly irregular. This situation 
IS a relatively common one m much of the research done in this 
field with which we are concerned The development of “distnbu 
tion free ’ statistical techniques may solve some of these problems 
The last chapter m this section describes the more important of 
these techniques 
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The systematic descrtpt.on o£ ‘^ese phenomen^^^^^^ 

tisu involves the recording of these sym P , „ „„ their 

fashion, classifying or categorizing them are 

quantitatne incidence and mterre cpneral statements ma) 

carried ont in a proper way. objecme and general 

be made about them 

QuaUtative Material Created by Social psychological Res 

Many of the techniques of research ‘’^’“^^^‘'pt'her Tyrabohc 
chologists have as their end pr net ^ example of this 

material The research interview is a prmcipa stimulaies ver 
technique Here the researcher, by as mg q certain 

bal behavior which he hopes wi P™J‘^^,3t,on5hips with others 
characteristics o£ the individual or o stimulated themes, 

Such variants of the interview as projectiv ' Experiments 

life histones, and the like are of a “‘“"lals that must 

m the laboratory and in the field also produce mater 
be submitted to systematic analysis , specifically stimu 

In research where the symbolic ,„a,cative of something 

lated. this material is usually taken to e ^en by a re 

beyond itself A particular statement, or researcher because 

spondent in an interview has significance attitude, value. 

It may be taken to indicate the presence analysis of such 

cognitive structure, or the like The svill make it possi 

statements therefore, must proceed in a conversion was 

ble to describe clearly to other phenomena to a specific 

made from a particular range of qua i 
genotvpe or hypothetical construct 


Catcgonzwg Qualitative Materials ^ qualitative 

When the social psychologist has j phenomena or as 

materials fmm records of natural s ^ ria«i[v the 


..z.^n the social psychologist na -odal phenomena or as 

materials either from records of natura ^^ant to classify the 

products stimulated by a research project e describe it m an 

Content into appropriate categories so t a categories is com 

orderly ^^ay This process of jpj^er term 

monly knoivn as content analysis or co - materials re 

u more frequently used m reference to q 
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kind of analysis done will limit the manner of statistical tieatment 
that will be permissible and effective 


IMPORTANCE OF TREATING SYMBOLIC 
PRODUCTS SCIENTIFICALLY 

Social psychologists are concerned with the analysis of qualita 
live material for two primary reasons The proper subject matter 
of social psychology consists, in large measure, of verbal and other 
symbolic behavior as it is found in soaety Methods must be devised 
to treat this behavior analytically But social psychologists do not 
confine themselves simply to recording and describing symbolic 
behavior as it is found in ‘real life they also construct situations 
designed to elicit symbolic behavior under more controlled condi 
tions In a certain sense, they “create” symbolic materials so that they 
may analyze them in keeping with the objectives laid doun in the 
design of these contrived situations 


Qualitattve Material Natural Phenomena 

When one stops to think of it, it is really surprising hou much 
of the subject matter of social psychology is in the form of verbal 
behavior The formation and transmission of group standards, 
values, attitudes, and skills are accomplished largely by means of 
verbal communication Education in the schools, m the home, m 
business, m the neighborhood, and through the mass media is 
rought about by the transmission of information and by the 
exercise of controls which are largely mediated through written 
or spoken words If one is concerned with problems of social organ 
ization, the situation is similar Supervision, management, coordina 
txon, an t e exertion of influence are principally matters of verbal 
interaction Social and political conflicts, although often stemming 
from divergent economic interests and power, cannot be fully 
understood without studying the words employed in the interaction 
o con icting groups, and the process of mediation consists largely of 
talking things out The work of the world, and its entertainment too, 
IS in no small measure mediated by verbal and other s^mbollC 
behavior 
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corded from nature, the latter is more commonly employed in the 
analysis of materials created by the research Coding is used espe 
cially to refer to the process whereby answers to interviews are 
categorized However, no universally accepted usage has emerged 
to distinguish one term from the other 

In an excellent discussion of the field of content analysis as it 
has developed in communication research, Berelson proposes the 
following definition Content analysis is a research technique for 
the objective, systematic, and quantitative description of the manifest 
content of communiCcttion (6, p 18) This is a satisfactory definition 
if It is interpreted liberally Communication should be thought of 
as any linguistic expression, and the restriction to "manifest con 
tent should be removed With these modifications, we have an ade 
quate designation of all the kinds of analysis of qualitative materials 
of interest to social psychologists In the subsequent discussion we 
propose to use the terms content analysis and ‘ coding’ inter 
changeably to refer to the objective, systematic, and quantitative 
description of any symbolic behavior 


Uses Made of Content Analysis 

The most detailed summary of the many uses of content analysis 
IS that of Berelson (6), who has developed a system of classification 
resulting m a listing of sixteen uses of content analysis of verbal 
material Although there are several alternative ways in which the 
work in the field could be classified Berelson's listing is quite satis 
factory We reproduce it here, with some of the studies cited by him, 
in the interest of standardizing terminology For a comprehensive 
bibliography of publications dealing with content analysis the 
reader is encouraged to refer to Berelson s book 

Three broad approaches to the analysis of symbolic materials 
are esign^ted by Berelson In the first, the researcher is interested 
primarily in the characteristics of the content itself In the second 
he tries to make valid inferences from the nature of the content 
to characteristics of the producers of the content or of its causes 
In the third, he interprets the content so as to reveal something 
about the nature of its audience or of its effects Any single study 
may or may not adopt more than one of these approaches 

CHARACTERISTICS OF CONTENT Interest in the first approach 
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will lead one to focus either on the -^stanttve namre or upon ^ 

form of the content Berelson hsts srx uses thich 

primarily with the substantive ^ ^ade among ma 

materials In the first two of these, compari 

terials produced at different points in ^ jp tj,e fifth 

materials coming from different sources -ontent is evaluated 

use, the observed substance of communica under the sixth 

against standards adopted by the substantive character 

heading, Berelson simply analvzed by researchers imesti 

istics of symbolic behavior are ofte V- 

gating reactions under controlled con ^ Many investi 

ro describe trends in 

gallons have been undertaken to eter j, pf such trends m 

periods of time If one is to esta is gn,ploy comparable meth 
communication content, it is necessa y ^ , successive points 

ods for sampling the total flow ^^™“jass,fication throughout A 
in time and to use the same system nnalvsis by Yakobson 

fairly typical example of a trend stu y is goviet Union They 
and Lasstvell (52) of May Day slogans i , a period 

found, for example, that May Day s og , gj-y symbols to 
of years from employing universal different 

'national ones Ojemann (40) studie ^ 

type of content He recorded trends in ar i Housekeeping 

appearing m the Ladie^ Home y,cre m the early 

and \sas able to show that articles o based upon scientific 

part of the century, much less frequen Y ^ j trend study 

authority * than they were by 1940 Still Here, 

n that in which public opinion is r natural phenomena to 

instead of relying upon the recor mg ^pbes the same qucs 
rcNeal trends, the social scientist repeate pf the population 

lions to comparable (sometimes . stud\ (H) Amer 

m order to detect changes of opinion before and after Pearl 

lean attitudes toward international a yi^ ^attitudes lot'ard the go' 
Harbor, and Cart'vrigbt's research (1 ) ^Vo^ld H 

eminent s inflation-control program tliroig 

examples of thi^ use , . ^ This use of content 

To trace the development «/ ' Uo'C It is mentioned 
analysis IS essentially the same as the on bas been done 

separately because a substantial amount o 
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specifically to detect trends in the publications of scholarly and 
scientific journals A good illustration of this kind is Allport and 
Bruner’s study (9) of the topics of research in psychology o\er a 
period of fifty years 

To disclose international differences m communication con 
tent With increasing interest in problems of international rela 
tions, social scientists are coming more frequently to study the sys 
tematic differences that exist among countries in the content of their 
major media of communication Two studies comparing Germany 
and United States may be cited Herbert Lewin (34) made a compa 
cable categorization of the literature of the Hitler Youth and the Boy 
Scouts of America in terms of their goals and justifications McGrana 
han and Wayne (35) compared the major themes of the most popular 
dramas appearing in Germany and America in the years 1927 and 
1910 In both these studies, substantial and rather similar dilTerences 
were found between the two countries Other studies comparing dif 
ferent countries have been made in terms of such media as radio, 
newspapers, and textbooks, and a few comparable interviewing sur 
veys have also been conducted in different countries There are 
obviously many difficult problems of sampling and of translation 
in making such cross national comparisons, but this kind of research 
has produced some of the most useful data now available for an 
understanding of national differences 


To compare media or “levels' of communication Students 
interested in understanding the role of the mass media m molding 
public opinion have made especial use of this type of content anal- 
ysis azarsfeld, Berelson, and Gaudet (30), for example, studied 
1 erences m partisanship among newspapers, magazines, and radio 
tto"® fl. «n.pa.g„ They found all three media 

iarn an ,h ‘he magat.net more ttrongly 

drferem ^ PS) analyzed the role of 

smdvrnl """P^P^* ■" 'he etty-s interracial relations by 

fhe" whue’'’’' d “n'"' P‘‘p""''H?fornr*arp dfffeTen^s'‘be»veen 

S^tf LhterXpST^IL^^^^ 

defendant s case Treatment of Tnanv 

1 , .n J a: t letnneni ot many other controversial subiects 

by the different media have also been compared ’ 

To comtruc, and apply commun.caUon standards Many 
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studies have been aimed can be made 

the media of communication. „,inst some sort of standaid. 

only by comparing acmal P" jor transmitting frivolous 

Criticisms of the media for being ■ ^ a, 

or trashy content, or for °'''="™P ‘[auhe media should be doing, 
least implicitly, certain standards of .prise it is possible tor 

It such standards can be made exp icit a P Jansmitted to pro- 

acontent analysis of thecomrnunica^ns^acm^^^^^^ 

vide objective evaluation of the me .cshnitv as an evaluation 

tent job of content analysis is ‘‘j “"^the standards employed. 

will still depend upon the accepta y 

Much ol the research in this fiel widely accepted 

ards proposed by investigators “PP^. .. “balance." Thus, 

cultural standards as "fairness," objec j’ . j, jg give a fair 
Sussraan, by assuming that radio has a ■ j eroup, 

and balanced presentation of every "’“i”'' analysis ot about 

to document charges ot bias throug i a c , -j,. ^ presidential 

thirty news programs on the major netwo being morally 

campaign. L found that "labor ° 

wrong five times as often as it was mor y b presented 

it was presented as being strong just as evaluating per- 

as being weak” (45, p. 210). British Royal Com- 

lormance against standards is provi e ^ ^ list of major facts 

mission of the Press (41). This group set Coal Board, 

concerning the first year's progress of t le events" in various 
They then checked the reporting of these a 
newspapers and revealed very meager cov b 
girded as socially important informatiori. Under this heading, 
To aid in technical research uses of content 

herelson has grouped two of the coding of free-answer 

analysis in contemporary social researc . t ^ people in groups- 
interviews and the analysis ot interaction a , surveys is found 

^ thorough discussion of the many uses ol m discussed 

m Chapter 1. The problems involved in the 
"t detail in Chapter 9. 

. ujjye in common a 
The next three uses of content to its substantive 

meus upon the form ol the content (m co ^lysis of propa- 

"xvre). The first of these is concerned with me 
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ganda The second derives from a practical inteiest in improving 
the intelligibility of written communications The third has been 
found most commonly among those interested in the study of htera 
ture 

To expose propaganda techniques Content analysis of propa 
ganda has quite often been designed to reveal the ways in which the 
propagandist pursues his objectives of influencing the public Berel 
son points out that two broad classes of techniques” have been 
studied the themes or appeals employed, and the "tricks of the 
trade Examples of the former are a study of British and German 
propaganda in World War I by Lasswell (26) and an analysis by 
\Vhite (48) of the values employed by Hitler and Roosevelt in their 
speeches just before World War II Lasswell concludes, for example, 
that the British stressed humanitarian ideals much more than did 


the Germans, and TVhite found, among other things, that 35 percent 
of Hitlers emphasis units' invoked the value of "strength ’ in con 
irast to only 15 percent in Roosevelt’s speeches A study by Lee and 
Lee (31) illustrates research on the second type of technique Here 
seven tricks of the trade were enumerated in the speeches of Father 
Coughlin Other studies have investigated more particularly the 
uiili/ation of emotional content Waples and Berelson (47), for 
example constructed an index of the incidence of emotional terms 


in vinous media during the 1940 presidential campaign and found 
signiPicintly more emotional content in material dealing with 
Roosevelt than m that dealing with Willkie 

To measure the "readability" of communication materials 
Early interest in the ability to grade materials on the basis of their 
1 1 iculty of comprehension was displayed by educators wanting to 
make textbooks fit the age and mental level ot different gioups 
of students Various schemes have been developed for this purpose 
The one currently enjoying the greatest popularity is that of Flesch 
( ) ccor mg to this scheme, two major components of "read 
case, which is measured by the number 
o s) a es per ^”d by the average length of sentence, and 

human interest, which is measured by the percentage of ‘ person-.! 
words and penonal sentences " Interesting problems of validation 
o is measure ave een noted, one critic showing that one such 
scoring system makes Kurt Koffka easier reading than William 
J-imes ° 
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Ttie stvlistic characteristics ol 
To discover stylistic fentiir . Extensively through con- 

literary prodnets have Ehc examination by Miles (37) 

tent analysis. Typical examples „netvv since tlie sixteenth 

ot the ratio of verbs to ®'‘'^^'i’"*'''ctinner MS) “f alliteration in 
century, the study conducted y resolve questions of 

Shakespeare’s sonnets, and invcsti^ chronology of a given author’s 
disputed authorship and of the corre 

works (21. 53). ..^..rrvNT The second major approach 

PRODUCERS OR CAUSES OF co^ , to learn something about 

to content analysis consists of the at c ” causes of the 

the nature of the producer or. more g material itself, 

symbolic material from the , access only to the com- 

In some situations, where the researc communicator directly, 

municated material and cannot stu y ^rher situations, 

this' method is used as a matter o “P symbolic behavior as a 
where a person can be induced to pro such behavior 

response to standard conditions, the c “ person’s own 

are often taken as a very acceptab c in ,gut analysis illustrate 

characteristics. The following four uses o ,o construct a 

various ways in which social * udje products. 

picture of the communicator characteristics of the com- 

To identify the intentions and olM and attitudes ol 

municators. In a number of studies the sis of the content 

communicators have been inferred ® these studies have 

of their communications. Unfortunate y. , assessment of how 

not had tests of validity that allow a ve^ illustration 

successful the inferences as to intentions a ^8) 

of this type of content analysis is the stu Y labor newspapers 

in which the treatment of A.FX.-C.1. • co ^as following 

"^as analyzed to reveal how closely loca analysis revealed 

^he official position of the labor press, which 

a predominant position of neutrality y j^gj-able inertia on the 
'vas taken by the authors to indicate a of the national 

pan of local leaders with respect to the P virtually impos- 

organization. Another study, in which vali 

sible, 1*9 fK.., T r,i of speeches , ,_,uciov These 


organization. Another study, in which vah a 

sible, is that of Leites et al. (32) of speec birthday. These 
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toward Stalin Sharp differences were found in the image ot Stalin 
revealed by the old Bolsheviks and by the other speakers 

To deteimme the psychological state of persons and groups 
This use of content analysis has perhaps been of most value to peo 
pie interested in the study of personality Clinical interviews projec 
tive tests life histones diaries letters and other personal docu 
ments have been analyzed for this purpose Allport (1) has sum 
marized well the various techniques and objectives of this type of 
research In one study Baldwin (3) recorded the frequencies with 
which certain themes were contiguous to one another within a sam 
pie of letters written by a single person The clustering of themes 
found in the letters was taken to indicate principal motivational 
and ideational clusters in the writer s personality Another type of 
content analysis which is designed to reveal something of the emo 
tional adjustment of a client during treatment is the discomfort relief 
quotient developed by DoUard and Mowrer (17) This quotient 
IS computed by dividing the total number of discomfort words bv 
the number of discomfort plus relief words combined Suggestive 
conclusions have been drawn from noting changes in the value 
of this quotient during the course of treatment and from comparing 
the value of the quotient for one client and another Quite a dif 
ferent type of study is that of the United States Strategic Bombing 
Survey (46) in which captured German civilian mail was analyzed 
to determine the effects of strategic bombing on civilian morale The 
letters were coded in various ways to indicate evidence of low morale 
poor health and anxiety These indications of demoralization were 
then statistically related to other characteristics of the v\riters such 
as sex date of writing tonnage of bombs dropped on locality v\here 
the letter was written and the like From the relationships thus 
established certain conclusions about the effects of bombing on 
morale were drawn Thus for example day raids were found to be 
more demoralizing than night raids and certain community disrup 
tions were found to be more demoralizing than others In yet an 
other example content analysis was used to reveal basic personality 
orientations toward leadership m a study by Sanford and Rosenstock 
(42) These authors developed a projective device in the form of 
cartoon pictures which can be administered in brief interviews 
on the doorstep They find that responses can be reliabl) coded and 
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est) about the situation m \\hich the content is produced Often 
these assumptions have httle empirical justification and the infer* 
ences about the nature of the audience are worthy of consideration 
only because direct observation of the audience is not possible 
The following three uses of content analysis illustrate this third 
approach 

To reflect attitudes, interest, and values ("cultural patterns”) 
of population groups In several studies of the content of mass 
media, it has been assumed that the material communicated through 
these media express or reflect the prevailing thought and mores 
of the population at that time Thus, Hart (22) analj’zed the content 
of popular magazines in the United States oter a period of years 
from 1900 to 1930 He found what he took to be evidence for a 
decline m the status of religion and an increase in toleration of sexual 
freedom during this period The primary basis for these conclusions 
were changes in the amount of attention given to the topic in the 
magazines and the incidence of indicators of approving attitudes 
Wolfenstein and Leites (50) have drawn certain conclusions about 
the American culture of today from an analysis of contemporary 
feature films In this study it is explicitly assumed that a movie is 
a sort of national day-dream’ which reveals something of the emo- 
tional life of the movie going public One of the more ingenious 
aspects of this study is the interpretation of American relations 
between the sexes based upon the prevalence of the “good bad girl" 
in American movies, a type not so common in films of other coun 
tries In all studies of this type, it is difficult to determine why the 
investigators assume that the content reflects characteristics of the 
audience rather than of the producers Often the assumption seems 
to be either that the content reflects both equally well, or that it 
reflects the audience because the producers are for some reason 
(perhaps by a sort of ' natural selection ) attuned to the audience 
To reveal the focus of a«enfion A slightly different assump 
tion underlies inferences to the audience in this use of content 
analysis Here it is assumed that there is a more or less approximate 
correspondence between the content ot the mass media and the 
cognitive content of the audiences exposed to the media (presumably 
because the media produce the cognitive content) If some Item of 
information or theme is stressed in the media at some place and 
time. It IS assumed that this will be salient in the thinking of the 
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population Research on reading and listening behavior indicates 
that this assumption can, at most, be only an approximation and 
that various people exposed to the same content through the same 
media may react to it in quite different ways Some examples of this 
approach may be cited Woodward (51) showed that the percentage 
of foreign news in American morning newspapers in 1927 rarely 
exceeded 10 percent, indicating presumably, that American people 
were only slightly aware of foreign events Festinger Cartwright, 
et al (18), in studying the circumstances under ivhich an anti 
communist rumor broke out in a local community, showed that in 
the period preceding the outbreak of the rumor there was a marked 
increase in the number of column inches devoted to the theme of 
domestic communism in the newspapers read by the local citizens 
Although these newspapers did not deal at all with the specific 
subject of the local rumor, the authors argue that the media set 
or reflected, an atmosphere favorable to the occurrence of the 
rumor Amheira (2) conducted a now classical study of what kind 
of world IS brought to the attention of listeners to soap operas He 
showed, for example, that the world of the daytime radio serial 
deals predominantly with themes concerning personal problems 
rather than public affairs Of 43 such serials, for example, 49 percent 
dealt with problems of courtship, whereas only 26 percent were 
concerned with public affairs 

To describe attitudmal and behavioral responses to communtca 
tions Berelson points out that there have been three ways in ^shich 
content analysis has been used to study the effects of communica 
tions The first consists in analyzing materials which were produced 
response to some specific communication Lerners study (33) 
of published reactions to The American Soldier is a good illustra 
tion of this type of analysis The second kind of investigation 
attempts to show empirical relations between the content of a com 
uiunication and responses to it Thus, Berelson (5) showed that the 
greater the frequency of certain political arguments in the various 
*^edia, the larger was the number of people who could recognize 
the argument On the other hand, he found a much wcaler relation 
^hip between ihe frequency of appearance of the argument and 
ticccptincc of It hferion (36) also attempted to relate characteristics 
of the content of media to peoples reactions to it in his stud> of 
hate Smith s war bond marathon during %Vorld M'lr II He noted 
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among other things, that fully half of Smith s material stressed the 
sacrifice theme and, in intensive interviews with people t\ho heard 
Smith, he traced through the personal meaning of this appeal to 
listeners In the third kind of analysis, direct inference is made to the 
effect of content without any reference to response data themselves 
In this way Lasswell and Blumenstock (28) analyzed the themes 
employed by communists in Chicago in the 1930 s and concluded 
that their propaganda was relatively ineffective because it ran coun 
ter to the fundamental values and mores of the citizens to whom 
it was addressed 

It should be apparent from this summary that content analysis 
has received serious attention in widely different fields of investiga 
tion Any evaluation of the theoretical or practical significance of 
this technique must be made, as in the case of any technique, m 
terms of the specific objectives set up for the research The work 
completed to date undoubtedly shows that this technique can be 
successfully applied to the solution of many significant problems 
Only future work can demonstrate its full potential and limitations 
It IS possible, however, from the experience accumulated to date, 
to set up certain standards which should be met in the process of 
analyzing symbolic materials 

Let us turn now to a more detailed examination of the process 
by which qualitative materials are converted into scientifically 
acceptable data and examine some of the principles that should gov 
ern this operation 


CONVERTING PHENOMENA INTO 
SCIENTIFIC DATA 

The recording of symbolic materials as found in life settings 
and the stimulation of them in contrived situations provide the 
analyst only with raw matenals Inspection of such materials may 
lead a sensitive person to certain insights and conclusions and these 
may be m a certain sense, ‘correct But, in the long run both 
scientific and practical progress require more than sensitive insight 
(though both can certainly make good use of it) To the extent 
that investigators cannot communicate to others how their insights 
are accomplished, the ability to achieve them is retained as private 



Analysis of Qualitative Material 435 

property of individuals These conditions would produce, at best, 

experts and not a body of know e ge rn„^ert recorded “raw 

The objective of content analysis “ , ,„ent.Hc 

phenomena into may be built up More spec.fi 

manner so that a body of ^nnducied so as (1) to create repro 
cally, content analysis must be susceptible to measure 

ducble or •objective - data, wh. h 2) some 

ment and be 

systematic theory, and (4) y & 
set of material analyzed 

The Problem of Objectivity 

suppose that an ‘-estigator has cM^cmd such mamn^^^^^ 

speeches made by a political uj the United Nations 

or the New York Times, e ^nondents in a sample m 

Security Conned, or "e^go abolt'constructing his descriptive 

verify them independently? Four aspects of this probl 
xo an .mcovcti m 

less there is “^bel the^^ 

butes can be found in the same materia , j worker during 

from an interview conducted with an industrial 

World War II „ „ 

tu like to -;“;:“;eTre'3:ru and 

goods should be shared wh y , , We should 

raw goods and poor '’""y''”' we 11 hare another war 
see that Germany gets u w ^ for ours 

Russia is fighting for her way people 

England IS fighting along wit us Churchill and Roosevelt 

against fascism-to be bow the people feel We 

and Stalin are great men my „,„ore The union s taught 

can t stay on our side of the pond an, more T1 
me that 



436 The Analysis of Data 


Literally scores of attributes can be found in this brief passage 
Let us list a few (1) number of words, (2) percentage of personal pro 
nouns, (3) attitude toward free trade, (4) perceived cause of war, 
(5) degree of confidence in the Allies, (6) degree of confidence in 
leaders, (7) attractive traits of leaders, (8) attitude toward isolation 
ism. (9) evidence of previous isolationism, (10) source of influence 
on attitudes, (11) implied values. (12) inclusiveness of cognitive 
structure, and (13) degree of approval of Allied war aims 

It should be obvious that many other attributes of this material 
could be listed and that disagreements about the true nature of the 
material could easily arise Objectivity requires, therefore, explicit 
specification of the variables (sometimes referred to as "dimensions” 
or ‘ types of attributes ’) in terms of which descriptions are to be 
made This is the first step in constructing the analysis outline (or 
code ) We shall return later to the question of horv these variables 
should be selected 


THE CATEGORIES FOR EACH VARIABLE Let US assume now that 
we have chosen some variable, perhaps one from the list above 
confidence in the Allies There remain many ways in which this 
variable may be broken down into categories We might deade 
to code all interviews (or any appropriate material) so that each 
has to be placed in one. and only one of the three following cate 
gories (1) High (2) Low, (3) Not classifiable m either It would 
however be equally possible to use seven categories (1) Unqualified 
confidence (2) Qualified confidence (3) Confidence and mistrust 
equally balanced. (4) Qualified mistrust, (5) Unqualified mistrust 
(b) Question not asked by interviewer, (7) Question asked, but answer 
not classifiable in above categories It should be apparent that 
if two independent analysts were to code the same material, one 
using t e rsi set of categories and the other using the second they 
'uT “"’7“'"“'' descriptions of the same phenomena 

s ou a so e clear that many other systems of categorization 
a e posst e xp icit specification of the system of categories used 
with each variable is essential for reproducible analysis 

the ope^tional definition foe each category Trio inde 
pendent analysts might agree to analyze the interv.ews for con 
fidence in the Allies and to employ the threefold system of cate 
gomatton and yet they stdl might not agree at all in their actual 
codings To be sure that they „,11 agree, they need explicit rules 
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cators Often this can be conveyed by establishing the "core mean 
mg’ or ‘ideal type’ of a given category and then defining its 
boundaries through examples of indicators which will be taken to 
fall on each side of the boundary * 

ADAPTATION OF ANALYSIS OUTLINE TO THE EMPIRICAL CONTENT 
The most logically constructed and theoretically elegant scheme 
of analysis will not produce objective results if it does not m fact 
fit the material being analyzed Try, for example, to fit the inter 
view quoted above into a system of classification designed to reveal 
the respondent s stage of psychosexual development Reproducible 
coding will be possible only when the system of classification is 
properly adapted to the material being coded 

Lazarsfeld and Barton (29) have suggested that m constructing 
the analysis outline for use with free answer interviews, there are 
two adaptations to the empirical material which should always be 
made The first of these they call adaptation to the structure of the 
situation Thus, in the analysis of "reasons given for a certain 
behavior it is desirable to ‘ build up a concrete picture or model 
of the whole situation to which the reports refer, and then locate 
the particular report within this ‘structural scheme’ ’’ (p 159) In 
constructing a scheme for analyzing reasons women give for buying 
a specific brand of cosmetics, one would set up variables referring 
to such things as sources of information, sources of advice, motives 
related to use of cosmetics technical qualities of cosmetics, anxieties 
about consequences on health, considerations of expense, etc An 
excellent example of this approach is the study of men’s preferences 
in suits, coals, and jackets, conducted by the Division of Special 
Surveys of the United States Department of Agriculture (10) In 
designing the interview and the scheme of analysis, these investiga 
tors obtained detailed data bearing on three general areas (1) cir 
cumstances that led to the decision to buy the most recently pur 
chased suit, (2) what men wanted their suits to do for them and (3) 
the values men brought to bear upon their suit shopping and the 
means by which they thought they could attain them Several more 
specific variables were set up m each of these areas 


icartwnght and Fest.nger (12 15) have demonstrated that people do 
employ categories of judgment whose boundaries may vary m preasion and 
that I cu ties of categorization as reflected m decision time increase as the 
malenal being class.Sed moves (tom ,hc core of the category toward the 
boundary ® ' 
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The second type of adaptation to the empirical material is 
“adaptation to the respondent’s frame of reference” (p. 162). The 
need for such an adaptation becomes vividly apparent whenever 
one attempts to apply a classification scheme assuming greater 
sophistication or difFerentiaiiori than in fact exists in the respond- 
ent's thinking. Cartwright (1$) found, for example, in studying 
popular conceptions of wartime inflation control, that characteriza- 
tions employed by technical economists could not be used in coding 
popular descriptions of war finance. 

This requirement that we adapt the analysis outline to the 
respondent's frame of reference has one important consequence 
often overlooked. Consider its implications for tsvo different tech- 
niques of interviewing— the free-answer question and the fixed- 
alternative question. From the free answer, the researcher obtains 
verbal material which he must analyze according to bis scheme of 
analysis. When, however, a fixed-alternative question is asked, this 
scheme of analysis is given with the question, and the respondent 
Is asked to code the answer which he would have given had he been 
allowed to talk fully. Under these circumstances, if the analysis 
outline does not fit the respondent’s frame of reference, the only 
alternatives open to the respondent are to refuse to answer or to 
indicate a categorization which is not accurate. Crutchfield and 
Gordon (16) have produced convincing documentation of this danger 
by following up a fixed-alternative question with a series of free- 
answer questions designed to reveal the respondent’s frame of ref- 
erence. 

Difficulties in getting an a priori analysis scheme to fit the verbal 
materials have sometimes led analysts to abandon efforts to construct 
3n analysis outline before studying the content of the material. The 
i^esults of abandoning these a priori considerations tend to be the 
construction of an outline whidi reflects only the superficial or phe- 
notypical similarities and differences among the elements of the 
content. Experience suggests that it is better procedure to start 
ivith an analysis outline and then to adapt it in a self-conscious and 
orderly fashion so as to make it fit the content being studied. In this 
'vay, it is possible to examine systematically the modifications in 
the a priori scheme which are called for. If these modifications are 
substantial, one may wish to condude either that the original out- 
hne was inadequately conceived or that the material chosen for 
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analysis was not m fact appropriate In either event the analyst 
modifies his original conceptions in an explicit and self conscious 
fashion 

We should consider at this point the attempt that several inves 
tigators have made to construct a standard or all purpose scheme 
of categories to be used in a wide \ariety of studies Some of these 
schemes are extremely phenotypic consisting of such categories as 
positive or negative affect Others however are derived from a 
more or less developed conceptual system Examples of these more 
genotypic standard schemes of analysis are Bales (4) categories for 
analyzing interactions in discussion groups and White s (49) cate 
gories for describing values employed in verbal materials There 
can be little argument about the desirability of having standardized 
schemes of analysis so that different studies can be compared It is 
probably no accident however that there are as yet relatively feu 
studies conducted by independent investigators which use these 
schemes To be satisfactory the schemes must fit readily both a gen 
eially accepted conceptual system and the specific content being 
employed in each neu investigation 


The Problem of Quanttficatton 

One of the major reasons for developing an explicit and objec 
live scheme of analysis is that it makes possible quantification and 
measurement (provided that certain additional requirements are 
met) After material has been submitted to a scheme of analysis 
meeting the four requirements of objectivity listed above it is pos 
sible to determine frequencies establish quantitative relations and 
generally m many of the operations usually thought of 
as measurement The abstract features of measurement theory are 
discussed m greater detail m Chapter 6 We shall limit our discus 
Sion here to certain common problems and practices encountered 
m most current work employing content analysis 

the UNIT or ENUMERATION The quantitative treatment of sym 
bolic materials requires that one specify dearly the unit m terms of 
which quantification is performed We shall refer to this as the 
umt of enumerauon In our earlier discussion we referred to an 
other unit-namely the recording „n,t-as that segment of the con 
tent ^^hlch gets labeled when the analyst codes the content It is 
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counts the number 

tion-the recording unit is exactly the same “ ' j 

An Illustration in which numler 

o£ a speech given by a public employed as an argument 

o£ times ‘ American military streng ,?„un,ent is taken both 

for a certain foreign policy In this case an argument 

as the recording unit and the jhe two units are 

But let us consider an example in ^ ™ ^ 

not identical One might characterize an enti 

aid as predominantly favorab = ,nehes of the 

poses o£ quantification count the enumera 

editorial In this case, a column me wou recording unit 

tion, whereas the editorial as a whole would b^^the record.^g^ 

Quite different quantitative resu “ j„ jhe latter exam 

of various units of recording and jhe sentence as 

pie, for instance, we might just as ^ ihe^number of favorable 
the recording unit so as to be ^,ght find that 

and unfavorable sentences in the editori 

slightly more than half of the 55 percent, let us 

als are favorable We should then conclude that 55 p 

say, of the sentences are favorable “ ^ ^^mfimh as the unit 
whole editorial as the recording uni ercent of the column 

of enumeration we might conclu j^we choose to employ 

inches of the editorials is favorable The units we ^ 
must be determined by the purposes o t e ^5tomary to take a 
In analyzing free answer >"“™ews it is custo J 

single respondent as the unit of enumera. 

tive statements are made concern g ,,,„i,.s however, have 

display a given characteristic Some resulting 

used the recording unit as the u confusion arises In 

confusion Consider an example m several reasons for 

an interview each respondent is a owe favoring a particular 

taking a given political P”'""";;:’'' “'n^as the recording unit and 
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respondents who mention that reason The results are now confused 
Suppose for example, that the number of reasons turns out to 
equal the number of respondents in the study \Vhat can one con 
elude from this fact^ Obviously one can conclude very little, because 
the same result would have been obtained if each respondent had 
given one reason or if one fifth of the respondents had given five 
reasons each This practice of using some segment of the content 
rather than the respondent as the unit of enumeration is some 
times defended on the grounds that the analyst is interested in 
measuring a climate of opinion ’ or “culture’ rather than char 
acteristics of individuals A convincing logic for this procedure, 
however, has yet to be developed 

THE SYSTEM OF CATEGORIZATION Quantification and measure 
ment depend not only upon a unit of enumeration, but also upon 
the existence of certain systematic rclalionsliips among the cate 
gones If the content is to be used as an aid to measurement, and 
if certain quantitative ireniments are to be employed, the categories 
of each variable must be related to one another in cernm definite 
ways 

Lazarsfeld and Barton (29), in an illuminating discussion of the 
logic of measurement, make a useful classification of the types of 
systems of categorization which aie possible m coding qualitative 
materials These may be referred to as (1) dichotomies (2) serials, 
and (3) variables 

A system of classification which employs dichotomies calls essen 
tially for a judgment of the presence or absence of the attribute 
in question Examples of such a scheme are a listing of reasons, or 
the counting of a certain kind of word phrase, theme, or value 
Here the coder looks at each recording unit and notes either the 
presence or absence of the attribute under consideration Reliable 
coding of this type requires that an explicit judgment of ' presence 
or absence be made Sometimes under the time pressures of a 
research project, the coder skims quickly through a body of material 
looking for certain indicators and noting their presence The logic 
of this kind of coding requires however, that every recording unit 
for which he does not note the presence of the indicator be taken 
to mean that the indicator is. m fact, not present A failure to be 
sure that an indicator is not present often is found to lower the 
reliability of such coding 
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respondents who mention that reason The results are now confused 
Suppose, for example, that the number of reasons turns out to 
equal the number of respondents in the study What can one con 
elude from this fact? Obviously one can conclude very little, because 
the same result would have been obtained if each respondent had 
given one reason or if one fifth of the respondents had given five 
reasons each This practice of using some segment of the content 
rather than the respondent as the unit of enumeration is some 
times defended on the grounds that the analyst is interested in 
measuring a climate of opinion or culture' rather than char 
acteristics of individuals A convincing logic for this procedure, 
however, has yet to be developed 

THE SYSTEM OF CATEGORIZATION Quaniification and m'^asure 
ment depend not only upon a unit of enumeration, but also upon 
the existence of certain systematic relationships among the cate 
gories If the content is to be used as an aid to measurement, and 
if certain quantitative treatments are to be employed, the categories 
of each variable must be related to one another in certain definite 
ways 

Lazarsfeld and Ilaiion (29) in an illuminating discussion of the 
logic of measurement make a useful classification of the types of 
systems of categonzition which are possible in coding qualitative 
materials These may be referred to as (1) dichotomies (2) sernls 
and (S) variables 

A system of classification which employs dichotomies calls essen 
iially for a judgment of the presence or absence of the attribute 
in question Examples of such a scheme are a listing of reasons, or 
the counting of a certain kind of word phrase theme, or value 
Here the coder looks at each recording unit and notes either the 
presence or absence of the attribute under consideration Reliable 
coding of this type requires that an explicit judgment of “presence 
or absence be made Sometimes under the time pressures of a 
research project, the coder skims quickly through a body of material 
looking for certain indicators and noting their presence The logic 
of this kind of coding requires, however, that every recording unit 
for which he does not note the presence of the indicator be taken 
to mean that the indicator is m fact, not present A failure to be 
sure that an indicator is not present often is found to lower the 
reliability of such coding 
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o£ categories to produce a better rnmoansons This problem 

make strictly satisfactory quantitative ,„d 
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they deviate, on the average, f™"’ , imbalance developed 
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column inches of type) It is apparent that these examples do not 
refer to psychological variables and that the process of coding con 
sists of little more than transcribing anstvers to the tabulation sheets 
If It were possible to employ true \ariables in the anal)sis of psycho 
logical material many mathematical operations not othenvise pos 
sible could be performed on the data (for example, treating different 
points on the scale as ratios) 

In treating categorized data quantitatively, the basic operation 
is that of counting This is true whether the system of categorization 
IS a dichotomy, serial or variable After the material has been 
categorized the usual procedure is to tabulate the frequenaes ob 
tamed for each category If the system of categorization is that of 
a dichotomy the frequency for any given category is usually calcu 
lated as a percentage of some total possible frequency Thus, one 
notes the percentage of all respondents interviewed who mention 
economic imperialism as a cause of war, or the percentage of all 
lalue statements in a speech which appeal to strength If the system 
of categorization is that of a serial or variable, the frequencies for 
each category may be noted and such measures as those of central 
tendency and of dispersion may be calculated 

MAJOR REASONS FOR DETERMINING QUANTITATIVE RELATIONS 

Basically, the social scientist i$ interested in quantifying symbolic 
material so that he can compare different sets of material and ex 
amine relations m a precise wzy He may wish to do these things 
for any of several purposes Let us consider bneHy some of the more 
common of these 


here are two basic kinds of questions that are raised m most 
studies (1) How do symbolic materials \ary over time? 
and (2) How do materials produced by different sources differ from 
one anot er? Many examples of both types of relationships were 
given in t e survey of uses of content analysis presented above In 
esia is ing trends over time and m comparing different kinds of 
materia s it is essential that the same system of categories the same 
operational definitions of the categories and the same units of 
recor mg an o enumeration be used in quantifying the mate 
rials being compared This requirement is sometimes difficult to 
mee when the several materials ar^ quite different m content If 
the frame of reference of the person producing the content changes 
over ime or i difi^erent frames of references are used b} different 
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may reveal something oi the nature f 

ample evidence that people may no ' source of informa- 

determinants o£ their behavior, this rmportant 

tion should not be ignored. Upterminine causation, 

in the ideal - the probto of deters ^ 

a variety of techniques will be • mentioned. For example, 

in the same study the two approac les j States government 

in studying the war-bond program oj “ted^S^^^^^^^ 
during World War II, Cartwng ( ) drive. In addition, he 
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examined the relationships Ae drive. Some 

ports of what happened to tecame Ly had been 

people reported that they 1^°“® “interviews showed also that 
personally solicited. Analysis o „t.,iiv solicited were much 

people who reported P” „eopk who did not report 

more likely to have bought bonds Wo types of evidence 

a solicitation. The agreement be " . tj^ j personal solicita- 

heightens one’s confidence m the ^ P 

tion was a causal determinant of causation will be deter- 

Under the most favorable conditions, c™ 

mined by independent raanipu ^Utation under bombing, 

pendent variables. The study of quantitative indi- 

cited above (46), approximates this ae g 

cations of demoralization m wn community where 

of bombing (measured in -n-ge droppedF" the ^hat 

the letter writer lived. This e P ^ppH not necessarily be 
manipulation of the independent variable need 
performed by the researcher himself. 


The Problem of Significance made of much 

One of the most serious “*““*“* ‘^s 'irthat The "finding" 
of the research employing content a y In reviewing 

have no clear significance for either t ^ilber of studies which 

the work in this field, one is stru y f^gj-joation with counting, 
apparently have been guided by a s ee j.gjj to meet all the 

Unfortunately, it is possible for a .r enumerated above 

requirements of objectivity and ,o theory or practice, 

v/ithout making any appreciable co 
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rectly upon this objective We shall consider here only i few of 
the special problems involved in using content analysis in determm 
ing causation 

The covariation of two attributes is commonly taken to suggest 
that there might be a causal relation between them When s)mbolic 
material has been analyzed in a quantitative fashion it lends itself 
to this type of study It should be noted that for this purpose the 
two variables need not necessarily be expressed m the same units 
It IS possible to assert for example that a reduction of income 
reduces personal optimism without measuring the two things in 
similar units 


For many purposes a causal analysis is undertaken by looking 
for covariation of attributes within the same body of content The 
contiguity of certain themes in written material was taken hv 
Baldwin (3) as evidence for a functional interdependence TIiiv 
method is quite common in the analysis of interviews in sample 
surveys It may be illustrated by an unpublished study conducted 
during World War II in which a substantial correlation was shown 
to exist between expressions of internationalism and statements 
indicating an equalitarian ideology This finding was taken to sup 
port (but not to prove ) the hypothesis that a basic ideology 
influenced specific attitudes 


Sometimes it is possible to demonstrate covariation between 
features of coded content and some external variable In interview 
surveys i is approach is quite common if we assume that such things 
as reported age sex marital status income and the like correctly 
reflect external variables The index of political predisposition 
constructed by Lazarsfeld Berelson and Gaudet ( 30 ) from reported 
nrnn'h" residence illustrates this ap 

fw 7 si^owed that certain combinations of these 

disnnsp i^espondents appeared significantly to pre 


be noted approach to discovering causation should also 

times able , H « that the analyst ts some 

, ftom the naLre o£ the 

rasons £o^h r r'spfndents are o£ten asked to state 

reasons £or them attitudes or behavior The coding o£ such reasons 



Analysis of Qualitative Material 449 

interviews would have been futile for P-P^^i.^ntUtem 
Another way of stating the -'I"'" f”, line must 

analysis is to asmrt that the of the mate- 

yield a genotypic rather than p ? to its superficial 

rial. If the content is classified P[, ]„rned of relevance to 
similarities and diSerences, little will he learned 

“pure” theory or "practicar ®PP'‘““^"' 1 sis outline must yield 

To insist that the variables J . ,1,0 coder must 

genotypic descriptions does not imp y, categories. Even if 

always place the content direct y attitude toward 

a variable is designed to reflect, ^«c^ntices on an attitude 

Negroes, the coder need not necessari .. ^cy unfavor- 

sJe ranging, let ns say. from and still 

able.” It is possible to employ the nhenotvpic terms 

have the coder categorize ''“P"”'” ce of certain stereotypic 
(perhaps by noting the ° procedure is employed, 

characterizations of Negroes). If th / solicit procedure for 

it is necessary for the analyst to ",Se^caIe. Which 

placing these phenotypic P reliable and valid classi- 

ot these two procedures will resul sophistication of the 

fication will depend largely “P®’' , reliabilities with more 

coder. Usually it is easier to obta g !„ being able to 

phenotypic categories. There is attitudes. When the 

list explicitly the ^ ^„lex and fubtle. it is often found 

content dealt with, however, is complex , are able to 

more economical to employ sophistj^caied ;™° ^1„1 

interpret directly the genotypic significance of the material 


Tl.e Problem of Generalization 

A, a rule the content analyst is not ■"•"“■'j,'",i;;"‘nTl^c^ 
conclusions or findings strictly to J c anal«i5 in order to 

Almost invariably he undertakes his ‘ jata than just 

tescal something about a more genera jacc and time) 

tliosc s)’mbolic materials (produced . a limited set 

with whicli he deals. However, genera i ^tadc legitimately 

of data to a more inclusive procedures follosscd. 

unless certain conditions arc met an it is consement 

in considering tlic problem of gencrahranon. 
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It IS an all loo common error lo equate ‘scientific" with “reliable 
and quantitative ’ Unless the findings of a content analysis have 
implications for some theory, however vaguely formulated, the study 
can merit serious attention only on the highly tenuous claim that 
some day the significance of the findings will become apparent 
For this reason, significant content analysis begins with some 
systematic problem whose solution will be determined by the specific 
nature of the data resulting from the analysis This problem may 
stem either from a desire to extend a theory or conceptual model 
to some new realm of phenomena or from a need to predict or 
control events for some practical end In either case the investigator 
must have an a prion conception of the variables that are relevant 
to his problem The purpose of the content analysis is to indicate 
the presence or absence of these variables in the ' real world,” some 
thing about the relative magnitude of the variables, and something 
about the relations among different variables 

In constructing the analysis outline, it is therefore necessary to 
compose it of variables which mirror the variables of the researcher's 
a priori conception Thus, when Hart (22) wanted to use magazine 
fiction to test the hypothesis that American culture changed during 
the first quarter of the century in a direction toward greater toler 
ation of sexual freedom, he had to put in his analysis outline van 
ables that would refiect “degree of toleration of sexual freedom " 
When the Division of Program Surveys (14) conducted an interview 
study to determine whether peoples plans for the use of their war 
bonds would predispose them to cash bonds for the purchase of 
consumer goods, the analysis outline used in coding the interviews 
had to provide dimensions for categorizing “plans for the use of 


From these examples it should be apparent that the value of a 
con ent ana ysis will depend upon the quality of the a prion con 
ceptuahzation It will depend, also, upon the adequacy vviih which 
is conceptim ization gets translated into the variables of the analy 
IS ou me ina y, it will depend upon having data to analyze 
which are appropriate to the variables of the outline Had Hart's 
magazine fiction presented no evidence whatsoever about toleration 
of sexual freedom, his hypothesis could not have been tested by his 
content analysis Similarly, if the interviews had not revealed any 
thing about people’s plans for thetr bonds, the analy SIS of the 
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proper procedures for drawing a P ^ ;.nown probability 

employed Each unit of the universe selecting units must 
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be independent of any correlations among sources, it the 

These requirements apply both to -lectmn of 

unuerse contains more than one sourc '.r selection 

IS respondents in a sur\e> or newspapers, e 

of content from any one source because of cor 

Let us illustrate some of the dangers selected 
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as our source a single newspaper an i la cmplo> s%ould 

content from it Certain proccdur<» i rat '' c\cr> issue 
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to distinguish two rather diffeient types of inference which may be 
involved in the process The first type rests upon the assumption 
that the materials analyzed are a representative sample of some 
specified universe of (actual or potential) materials The need to make 
this kind of inference derives from practical considerations of con 
ducting research Money and time will be saved if the description 
of a small sample can be taken as a safe description of the complete 
universe The second type of inference rests upon the assumption 
that the discovered relattons between certain conditions and cer 


tain consequences are universally true In this type of generalization 
it is asserted that whenever and wherever the specified conditions 
obtain there will follow the specified consequences and no assump 
tion need be made that the quantitative incidence of certain condi 
tions found in the sample will also be found in the universe 


ASSURING THE REPRESENTATIVENESS OF A SAMPLE In principle 
a satisfactory system for sampling materials m a content analysis will 
consist of four elements (1) specification of the universe to which 
generalizations are to be made (2) a guarantee that every unit of 
the universe has a known probability of inclusion in the sample 
(3) a procedure of sampling which is independent of correlations 
among units of the universe and (4) a large enough sample to 
provide a sufficiently small random error of sampling 

The theory and practice of sampling have been extensively 
developed m recent years and the reader is referred to Chapter 5 
for a systematic discussion of the general problem Here we shall 
limit our discussion to some of the more special problems encoun 
tered in applying sampling theory to content analysis 

Consider first the problem of specifying the universe of sym 
bohc materials to which generalizations will be made In any given 
study the universe that should be selected will depend upon the 
purposes of the investigation If for example the purpose is to 
compare the editorial content of a given newspaper against some 
standard so that legal action may be taken (as in detennin.ng the 
Axis line of native fascist papers during World War II) the universe 
under consideration should be all editorial content appearing m 
all issues of that one newspaper over a certain period of time In 
this case the problem of samphng ,s to guarantee that the selected 
specimens of content accurately represent the total output of the 
newspaper If however the purpose of the study is to compare 
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to do mth concept development, hypot es -nnnection Our 

design, etc. and cannot be disced u causation m 

earlier discussions of the problems findines treat some of 

content analysis and of produang signi .i-.pd to the analysis 

die more imrorunt considerations espeaally m the y^_^ 

of quaUtauve materials Once a „£ repl.mung 

tentatn ely formulated, the resear^tas ec , apparenil) 

the study! seeking limiting conditions, and analjrmg PP 
exceptional’ cases ,v,p difference between 

It may be useful at this point to — 

two major types of dieir study of cotmg be 

^isfeld, Berelson, =>"<1 Gaude^ 

havior in Erie County, Ohio m IW , residence 

such as religion, soaoeconomic status, nart\ ratlier than 

predisposed a voter to cast his ballot for ews in 

another This ’ finding was based ” P , dents of Erie 

the county A generalization of this fin g , employed uas 

County resu upon the assumption county cannot 

tepresentatne of the entire county Si jn the United 
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sample This possibility may be dramatized b> an extreme instance 
Assume that we order the issues as they appeared in time and that 
we sample one out of every seven issues of the paper Suppose that 
by some chance procedure we happen to select Sunday as the starting 
point Now our entire sample will consist of issues appearing on 
Sunday and we shall have a disproportionately large inndence of 
features which appear only on Sundays Obviously, this would be a 


bad sample of the total output of the paper 

Many kinds of orderly fluctuations may be found in the content 
from a given source Mintz (39) has described three major types 
and has investigated some of the problems of sampling associated 
with each The first of these he calls primary trends The news 
paper treatment of some topic will often show a gradual build up 


over a period of days If the procedure of sampling happens to select 
disproportionately from the beginning or ending phases of this 
trend estimates of the amount of space devoted to the topic will 
be correspondingly too small or too large It is clear, of course that 
such trends are not always linear The second type of orderly fluctu 
ation IS a cyclical trend An illustration of this kind is the weekly 
schedule of certain topics m a newspaper or the regular scheduling 
of certain types of radio programs at certain times of the day If 
these peaks are either over or undersampled corresponding 
errors in estimating the total universe will result The third type 
of orderly fluctuation is where there are compensatory relations 
between adjoining units Take for example a nexvspaper in which 
there is a tendency to give a continuing news story a big play on 
the first day but little space on the next day If the sampling pro 
ce ure were to select issues of the paper appearing on alternate 
ays t ere might result systematic errors in estimating the total 
universe All these dangers can be nnnimized by following proce 

onhe Mh'Ir'''*' sapling unit is independent 


11 noted that under certain conditions there 

hie' “ Stratifying the universe That is to say 

whenever there is reason to believe that certa.n classes of units may 
be ^re homogeneous than the tntaf universe these classes may be 
designated and the requirement set up that the sample contain a 
proper proportion of units from each class In these circumstances 
the selection of each unit should of course still be done in a random 
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conflicting predisposing factors (such as a rural Catholic or a low 
income Protestant) displayed various symptoms of conflict in making 
a political decision For example, they took longer to make up their 
minds and showed more vacillation in their party preference The 
theory that conflicting forces or cross pressures produce such symp 
toms of conflict may be proposed as a universal "law" which should 
hold wherever or however such cross pressures are exerted The 
truth or falsity of this theory does not depend upon the representa 
tiveness of the sample employed in the study, any valid exception 
to It found anywhere would be sufficient to require a modification 
of the proposition 


CONSTRUCTING THE ANALYSIS OUTLINE 

The preceding discussion has examined the major systematic 
principles involved in converting phenomena into scientific data 
In conducting research, more is needed, however, than an under 
standing of these fundamental principles The success of any project 
will depend upon the degree to which these principles are expressed 
as actual procedures Let us turn then, to a consideration of some 
of the more concrete and detailed procedures involved in carrying 
out investigations that employ content analysis 

How, specifically, does one go about constructing an analysis 
outline? Six steps for arriving at a satisfactory analysis outline may 
be indicated These steps are intended to suggest clusters of inter 
related decisions which the analyst must make They are points at 
which It IS useful to check the emerging outline against the general 
principles listed above 


Step 1 Specify Needed Data 

In laying out an analysis outline, it is essential that the investi 
ptor have clearly in mind specifically what data are required by 
his total research design Ordinarily, he will encounter less difficulty 
in the long run if he is able at this point to work out his plans in 
sufficient detail so that he can tell what form his final tables will 
take If. for example, his design calls for testing m a set of inter 
views the relationships between information about international 
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Device-^vhit is the rhetorical or propagandistic character 
of the communication’ 


Step 4 Fill in Categories for Each Variable 

There are many systems of md uportL ob 
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used, to be sure that it is hmes, reasons, arguments, 

categories which call for listings o ’i]„ vulnerable to this 

sources of influence, and the like seem e p , where people 
cype of error The following ^'^-^““"//^^example place 
were solicited to buy war bonds is no ^ categories 
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one categorize a fanner ivho was soucite confronted with a di 
In constructing categones, one ‘a ° , specific meaning, 

lemma It a category is too broad, it conv ) £vpp, the 

hut if It IS too Lrrow, the coded ;coug.. use of 

raw” material One resolution of this foe classifying 

grouped categories Thus, ° desienate such broad calc 

reasons for buying war bonds mig „,,.,nric ” * National cco 
gones as “Personal financial,” ' Nationa p . ^ jppre specific 
nomic," etc Then, under each heading ther b good 

rategories. such as “Bonds are safe rr"®™' ■ ^ j^om lempta 
of mierest,” “Money imestcd m one findings fro^ 

“ons of spending," e.c In the interpretation of fm P 
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analyzing interviews, these variables will be used to classify not only 
various features of the answers to questions about the respondent s 
psychological make-up but also such matters as his age. income, mari- 
tal status, and other demographic or behavioral characteristics. In list- 
ing the variables to be included in the outline, care should be taken 
to assure that all information needed on the punch cards is placed 
on some variable. Thus, the outline should contain provision for 
coding the name of the study, the number of each enumeration unit 
(interview, issue of a newspaper, etc.), the name of each coder, and 
any information relevant to tests of reliability or other statistical 
treatment. 

In the literature on content analysis of communication mate- 
rials, certain types of variables have been employed rather fre- 
quently. These have been summarized by Berelson (6) under the two 
broad headings of “What is said ’ and “How it is said.” The vari- 
ables listed by him under each are given here, in order to indicate 
some of the kinds of variables that one might profitably employ. 

A What IS SAID 

1 Subject maiier-whai is the coinniunicaiion about' 

2 Direction— IS the treatment fatorable or unfavorable to- 
ward the subject? 

S Standard— what is the basis (or grounds) on which the 
classification of direction is made? 

4 Values— what goals are cxphciUy or implicitly revealed? 

5 Methods— what means or actions are employed to realize 
goals? 

6 Traits— what characteristics of persons are revealed? 

7 Actor-who initiates actions? 

8 Authority— in whose name arc statements made^ 

9 Origin— what is the place of origin of the communication? 

10 Target— to whom is the communication particularly 

directed? 

B How IT IS SAID 

1 Form of communication— is u fiction, news, television, etc ? 

2 Form of statement— what is the grammatical or syntactical 
form of the unit of analysis? 

3 Intensity-how much strength or excitement value does 
the communicaUon have? 
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Patience ot several people VNho arc supp 



458 


The Analysis of Data 

study, the investigator may utilize each level o£ classification lor 
different purposes 

I£ the analysis outline contains a considerable number ol van 
ables. It IS likely that rather similar systems o£ categories will be 
£ound among these variables In the interest o£ coding speed and 
the reduction o£ errors, it has been £ound desirable to establish 
certain consistencies in the way in which the categories are arranged 
The University of Michigan Survey ResCsarch Center (44) has estab 
lished certain conventions which it follows in categorizing free 
answer interviews For example, the category ‘ )es*' is always gi'cn 
the code number "1, the category ‘ no' number "5,” “don’t know 
number 9, none number ‘0,’ etc In a similar way, it has been 
found desirable to standardize the numbering system for scales so 
that they all progress in the same direction from positive to negative 
or from high to low With such standardization, the coder can soon 
categorize material almost automatically 

When the analysis outline has been completed, with all the 
categories defined, a manual of instructions for coders should be 
written giving these definitions in clear operational terms 

Step 5 Establish Procedure for Umtmug the Material 

We have defined above three kinds of units which must be dealt 
with m any content analysis the recording unit, the context unit, 
and the unit of enumeration The specific working definitions to be 
used in the study should be established at this point in such a 
fashion that various coders can all unitize the same material in the 
same way These definitions should be written down as a part of 
the coding instructions The selection of definitions of these units 
should be guided by the same theoretical framework that determines 
the rest of the research design “Practical ’ considerations of coding 
efficiency and reliability should not be ignored in deciding such 
things as the size of unit, but valid coding depends upon the 
theoretically correct selection of units whose categorization can 
properly be taken to indicate some significant feature of the 
material 

The most common recording units in communication research 
are (1) a single word, (2) a theme, usually consisting of a subject 
and predicate or some larger unit which can be condensed into a 
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USING THE ANALYSIS OUTLINE 
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same source, it is desirable in some kinds oE economic surveys to 
employ a ‘'spending unit" as the unit of enumeration. Information 
may be obtained separately from each individual, but it would be 
pooled into one “spending unit” in order to construct a single unit 
for purposes of computing frequencies, means, and distributions. 

Step 6. Try Out the Analysis Outline and Unitizing Procedure 

After the analysis outline and the unitizing procedure have 
been developed, they must be applied to the content in a prelim- 
inary way in order to discover what modifications are needed. Ordi- 
narily, this trying out of the coding procedures is also used as a 
training period for those people who are to do the final coding. 
When this period is ended, the analysis outline should be fixed in 
its final form and the coders should be “set” in their use of the 
coding procedures. 

This stage has been standardized at the University of Michigan 
Survey Research Center (44) in a procedure known as the “Round 
Robin.” A random set of materials is collected, and each coder codes 
it independently. All disagreements among coders are noted and 
used as a preliminary check on the reliability of coding. These dis- 
agreements are also examined to see what improvements in the 
analysis outline should be made. It is not uncommon to make sub- 
stantial modifications at this point. Variables of the analysis outline 
which do not fit the material well will need to be redefined or 
eliminated. Systems of categories which are cither not exhaustive 
or not mutually exclusive will be detected and revised if the Round 
Robin is done well. Those variables made up of listings can be 
expanded so that few addiUons to the list will be made after “pro- 
duction coding gets under way. And, finally, the system of notation 
on the coding sheets will be checked to determine whether it is most 
convenient for rapid and automatic coding and whether it will facili- 
tate tabulation as much as possible. 

When the Round Robin is completed, the whole coding pio- 
cedure should be frozen so that the entire set of content will be 
coded in the same way. Any modifications of the analysis outline 
after final coding has begun must be made retroactive to all mate- 
rials coded prior to the change. Obviously, much time could be 
spent in making such changes if they were to occur very often. 
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coding demands, therefore, that the outline be used in the same 
way (the same operational definition of categories, the same frame 
of reference, the same degree of differentiation, the same level of 
attention to details, etc.) throughout the entire coding operation. 
A person who is easily satiated with repetitive work will conse- 
quently not make a good coder for a very long period of time. 
Studies of satiation by Karsten (24), Kounin (25), and others have 
shown that the very requirements of sensitivity, motivation, and 
deep involvement in the task tend to hasten satiation if the meaning 
of the task is that of sheer repetition. Such satiation produces errors 
and variability in the application of the analysis outline to the mate- 
rials. Unfortunately, since no good test of susceptibility to satiation 
has been developed, there is little that can be done at present to 
minimize this problem through differential selection of coders. It 
would appear, however, that people who view the task of coding 
as routine, menial work will view' it as mere repetition of the "same" 
activity and will, as a result, be satiated more readily. 

If the coding is to be carried out by a team of coders, it is neces- 
sary that they all come to apply the same definitions and frame of 
reference to the coding. The achievement of such a common ap- 
proach to the task is best accomplished through group discussions, 
the Round Robin, and the sharing of difficult coding decisions. A 
coder who is uncommunicative or ego-defensive will, therefore, not 
contribute well to this purpose and will probably heighten the 
unreliability of coding. Again, it must be pointed out that no very 
satisfactory objective test of these personality traits now’ exists, and 
that selection for these traits is difficult. 

In present practice it appears that good coders are discovered 
mainly through a process of selection “under fire,” and that some 
provision might well be made to begin wiih a somewhat larger staff 
of coders than will survive” to the end of the project. 

Large research organizations who maintain a permanent coding 
staff have found it difficult to maintain the same people over a period 
of years at a high level of morale. Sensitive and intelligent people 
who are acquainted with the concepts of social science rarely find it 
satisfying to make a life career of such repetitive and routine work. 
Rarely can such a person work full-time at such a job for more than 
a year or two without considerable demoralization. Much better 
morale seems to result when the arrangement is for part-time or 



Analysis of Qualitative Material 465 
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might subsequently arise are ^sorked out A running record of coder 
disagreements is essential as an aid m revising the outline and as 
an indication of when the Round Robin may be safely terminated 
When, by actual performance, the coders have demonstrated their 
ability to code reliably m the way called for by the research design, 
‘real coding may finally begin 


Mechanics of Coding 

The orderly processing of materials requires that regularized 
procedures be established for their storage, assignment to coders, 
and recording It has been found convenient to package together a 
collection of materials (perhaps ten interviews or ten issues of a 
newspaper) and to have a coder * check out” a package at a time 
When one package has been coded, it is relumed to a central storage 
place and a new package is taken In the assignment of materials to 
coders, it is desirable to randomize the materials so that any sys 
tematic biases among coders will not appear as trends or correlations 
in the coded data An orderly assignment of materials to coders will 
also assure, of course, that all materials get coded once and only 
once Similar care should be exercised m collecting and storing the 
sheets upon which coding has been recorded 

Even after the Round Robin has been completed and final 
coding has begun, it may be found necessary to add new categories 
to some of the variables of the outline Or it may be discovered 
that too many items are falling into such catch all categories as 
other reasons If a coder comes across a recording unit for which 
there is no category, he should bring this case to the attention of the 
coding supervisor The supervisor should assess the merit of adding 
a new category by determining whether the new category would 
be meaningful within the rationale of the system of categories m 
volved by ascertaining whether the case in point could not just as 
well be placed in an existing category and by judging whether the 
new category would be used frequently enough to warrant its 
separate designation outside one of the catchall categories If the 
decision IS made to modify the analysu outline, this change must be 
made on the outline being used by all coders Furthermore, some 
procedure must be established to guarantee that the coding of all 
previously coded materials is modified wherever required 
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records should be tabulated separately for each coder 
yield a measure of his reliability as a coding instrument The inte 
pretation of differences among coders should be made judicious y, 
of course, because it is possible that the coder suth the greatest 

number of disagreements might he the most ‘ sahd coder 

The proportion of the material t\hich should be check coded 
will depend upon the uses that are to be made of it For purposes 
of training and of maintenance of constant standards, there is some 
point in check coding a relatively larger proportion earlier m the 
coding process and to taper off as coding becomes stabilized In order 
to construct a measure of reliability, it is best to employ a random 
sample of all materials In some cases, where any error is deemed 
serious, it may be desirable to check code the entire set of material 
and to tabulate only the pooled judgment 


SUMMARY 

The fundamental objective of all content analysis is to comert 
phenomena (le, symbolic behavior of people) into scientific data 
We have specified four charaaeristics which scientific data must dis 
play (1) objectivity and reproducibility, (2) susceptibilit) to measure 
ment and quantification, (3) significance for systematic theory, either 
* pure or ‘ applied, and (4) generalizability 

In constructing an analysis outline for a given project, it will 
be useful to organize the work so that it consists of six steps, or 
clusters of decisions At each of these points the de\eloping outline 
should be checked against the formal requirements for scientific 
data These steps are (1) specifying needed data, (2) mapping out 
plans for tabulation, (3) laying out the skeleton of the outline, (4) 
filling in categories for each variable. (5) establishing procedure for 
unitizing the material, (6) trying out the analysis outline and 
unitizing procedure on a sample of the material 

The successful use of a well developed outline depends upon the 
selection of capable coders, effective training of them in the outline 
being used, and the establishment of good supervision so that proper 
procedures of coding are followed 

Experience over a number of years with content analysis reveals 
that when technically well executed it can be a most \aluable tool 
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CHAPTER ELEVEN 


Theory and Methods of 
Social Measurement 

Clyde H. Coombs 


• riaia is a function of two 
What one "finds and how this information is 
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collected. Some methods of coUecti g methods, 

teristics of behavior to exhibit , eoHecting data impose 

Or, in opposite terms, some “j, do not. Obviously, 
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be inferred to be properties the information is 

The method of with certain properties, 

and may or may not endow this in , ^ with properties 
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cannot be inferred to be a character: . ^ analysis- 
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data with respect to the amount and kind of information each 
method contains about the behavior m question as distinct from 
that imposed Similarly, it becomes desirable to study the various 
methods of analyzing data in teims of the characteristics or proper 
ties each method imposes on the information in the data as a 
necessary preliminary to extracting it 

All of this IS a part of measurement theory, a subject of greater 
concern in the social sciences than it is in many other domains of 
knowledge Measurement in the physical sciences usually means the 
assigning of numbers to observations (a process called ‘ mapping ), 
and the analysis of the data consists in manipulating or operating 
on these numbers The social scientist taking physics as his model, 
has, frequently, attempted to do the same It is the thesis of this 
chapter that the social scientist who follows such a procedure will 
sometimes violate his data 

WHAT IS MEANT BY MEASUREMENT 

The objective of this first section is to show that the theory 
of measurement consists of a system of distinct theories, each cor 
responding to what may be called a level of measurement, and that 
a given set of data may satisfy (permit the valid use of) some of these 
levels of measurement but not others This first section will be 
concerned with an incomplete generalization of the logic of measure 
ment, with examples at each of the levels discussed This will be 
followed by a section on a theory of data in which the distinction 
between collecting and analyzing data will be discussed in terms of 
certain abstracted invariants of the behavior of individuals This 
theory of data is an effort to construct a framework in terms of which 
all methods of collecting and analyzing data may be unified under 
a general system In the final two sections, methods of collecting 
and analyzing social psychological dan will be discussed in the 
context of this general theory of measurement and the theory of 
data 

Throughout this chapter, the major emphasis will not be on the 
application of specific techniques This type of material is available 
throughout the literature Instead, emphasis will be placed on the 
assumptions underlying the several techniques and their structure 
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variety of other constancies are to be anticipated The mappings o 
individuals into such classes as athletes, politicians, ' 

are examples of stereotyping and constitute nominal scales Cultura 
and educational factors affect the construction of such nomina 
classes, creating new ones, dismembering old ones, and creating 
relationships among classes The relationship between nomina 
scales and perception is so close that whether or not an individual 
perceives and what he perceives is dependent in the first place upon 
the existence of nominal classes and in the second place upon the 
range or spread of the classes 


The Partially Ordered Scale 

Sometimes the objects in one class of a nominal scale are more 
than just diSerent from the members of another class— they may bear 
some kind of a relationship to them One such relationship is that the 
members of the one class are more of something than the members 
of the other class and u is meaningful to say that the relation 
greater than (>), in some respect, holds between the members 
of one class and the members of the other Given a number of 
equivalence classes, if such a relation holds between some pairs of 
classes the result is a partially ordered scale For example, suppose 
one wants to measure something to be called socioeconomic status 
Let us also suppose, for the sake of simplicity of illustration, that 
this attribute is made up of income and educational level If indi 
vidual A has more income and at the same time more education than 
individual B, he can be said, then, to have a higher socioeconomic 
status than B, (A > B) Further, if B has more of both constituent 
attributes than a third individual C, then not only is B > C, but 
also A has higher socioeconomic status than C, (A > C) It is 
apparent, then, that this relation is transitive It is asymmetric, 
houever because il A > B, then B > 

Suppose now that there were some fourth individual, D, who 
had less of both attributes than the first individual. A, and more of 
both attributes than the third individual C It could also be said 
then, that, with respect to socioeconomic status, > D > C But 
suppose at the same time that, although D has more income than 
B, he has less education This poses a problem It is not immediately 
clear whether B > D or D > B with respect to socioeconomic status 
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Suppose one were interested in measuring the popularity of 
individuals and one observed ‘ who likes whom ” The relationship 
A > B means that individual A is liked by the same people who 
liked individual B and some others in addition The data might 
be expected to yield a partial order To construct a simple order, 
one needs only to assume that being liked by one individual is 
equivalent to being liked by any other individual Then immedi 
ately the popularity of A is measured by the number of people who 
like A and at least an ordinal scale has been constructed (a certain 
abstract equivalence to the procedures of mental testing should be 
evident here) 

The process indicated here is not an unusual one It is because 
of precisely such procedures that operationism (2) was given so much 
attention Operationism is the principle that the meaning to be 
given to a concept resides in the operations which give rise to the 
measure of the concept Aside from the philosophical reservations 
one might have about operationism, the warning must be given in 
passing to avoid operationism in reverse By operationism in reverse 
IS meant endowing the measures with all the meanings associated 
with the concept In the preceding example the concept of socio 
economic status would mean income, with a unit of education 
counted as so many dollars A different operational definition 
would be a different concept— the equivalence of education to dollars 
is part of the definition 

Attempts were made above to construct an ordinal scale by 
mapping a partially ordered attribute into a simple order We saw 
that this mapping required a decision to be made on the exchange 
rate between the components and that every different exchange rate 
uould result in a different simple order for a large population If 
the experimenter insists on mapping a partial order into a simple 
order, the best he can usually do is to set up an objective com 
mumca e rule for the substituting of components and impose it 
on the data 

One alternative procedure is to study the partial order itself 
It IS possible, for example, to build a partial order by combining 
two or more simple orders in certain ways Thus there is the sug 
gesiion that a partial order may be decomposed into simple orders, 
a process analogous to factor analysis This problem appears solu 
ble (10) and constitutes a generalization of multiple factor analysis 
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to nonmetnc systems The determination of the dimensionality of 
a partial order (corresponding m principle to the rank of a correla 
tion matrix) already appears possible, but routine procedures for 
decomposing a partial order into alternative sets of simple orders 
(corresponding in principle to the rotational problem of factor 
analysis) have still to be developed 

A “natural ’ ordinal scale is obtained tv hen the raw data them 
selves contain the relation “greater than*' for all pairs It is remark 
ably difficult to find an example of a simple order among social 
psychological variables, since partial orders are almost universal 
Let us suppose, hmvever, that ive wish to measure the “authority” 
of some military personnel of various ranks and that instead of 
deriving the scale from military regulations we shall derive it from 
observational data Let us take “authority” to mean 'who bosses 
whom" and represent this by the familiar relation “greater than 
If individual B commands a platoon and individual A commands 
the company which includes that platoon, then A B By such a 

means a simple order along a chain of command can be formed 
There is some difficulty, however, about the definition of an equiv 
alence class corresponding to a specific ordinal position, a problem 
at the level of the nominal scale Consider two individuals, C and D, 
wch commanding a separate group of twenty men If there is a one to 
one correspondence between the members of one group and the mem 
bers of the other such that the men are potentially interchangeable 
then C and D can be regarded as bossing' the same men and hence 
members of an equivalence class If the men are not interchangeable 
vvithin pairs, however, the authority structure becomes a partial 
order again Let us imagine, however, for the sake of simplicity, 
that corresponding elements are interchangeable We have then a 
simple order of “authority ” 


The Ordered Melrtc Scale 

In the three scales discussed hcrciofore— nominal, partially 
ordered, and ordinal— the dements of the system vvere classes of 
objects and the relationships were relationships of equality and 
{^renter than It stiould be observed that nothing Ins been siij 
about a concept of distance bctuccn classes The formal introduc 
tion of distance by means of a distance (unction is beyond ij,p 
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scope of this chapter, and the concept of distance between classes 
will be dealt with here on a purely intuitive level In the scales 
discussed so far, all relationships such as “greater than” were be 
tween objects, which were stimuli or individuals Consequently 
there was a complete absence of such a relationship between any 
pair of distances between objects Thus, although A may have been 
observed to be greater than B, and B greater than C, nothing has 
been introduced concerning whether A was greater than B by a 
larger amount than B was greater than C From this point of view 
the scales introduced up to thts potnt may be regarded as nominal 
scales with respect to distances between classes of objects 

This introduces the concept of composite scales in which there 
are “"d "> which, in addition, there 
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ordered metric, there must be information in the data which lea s 
to such conclusions as that the difference in authority between a 
corporal and a buck sergeant is either greater than, less than, or 
equal to the difference in authority between a private and a corporal 
If the raw data consist of “who bosses whom, it mig t e poss 
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corporals, and by three buck sergeants. With respect o e ^ 
ment of authority, as defined here, the master sergean 
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scope of this chapter, and the concept of distance between classes 
will be dealt with here on a purely intuitive level In the scales 
discussed so far. all relationships such as “greater than" were be 
tween objects, which were stimuli or individuals Consequently 
there was a complete absence of such a relationship between any 
pair of distances between objecu Thus, although A may have been 
observed to be greater than B, and B greater than C, nothing has 
been introduced concerning whether A was greater than B by a 
h? ‘hau C From this point of vL 

scales w^h" a P as nominal 

scales with respect to distances between classes of objects 

are reUtmm"her“ “f composite scales in which there 

are relation ■" ‘•Edition, there 

s:::.ei:;rsc^rd"m"a;''Ltrnrh^““" 

and the ordinal scale .si oTdered^ornlucat''' 

.f, .n aSot a of these scales that 

between classes, then a higher** 

When a relation of “greater than’ hilH “ achieved 

between adjacent obiects on an I* distances 

partially ordered scale 

such distances, the scale is an a ' bolds for all pairs of 

For the sake nf “’■‘‘'■■cd scale 

between those tsvo I'ells' b'ulsh'*'ir“T'' distinguish 

ordered metric scale By an ordered “S^'ber as an 

It can be said of anv t i is meant a scale of which 

for at least some inte s be, ‘bat a > b > ^ and also that 

euher T 

Signifies the distance from ; to^ ^ ^ > i;a where m general, JK 

To illustrate the idea of an , 

previous example of a simni j nietnc we shall take the 
build It into an ordered metric*^s,^l "authority” and 

It will be best to give names , , ‘^‘’"'’on’cnce of discussion 

ordinal scale Let iliese be, m ord° ^ ^^^'ivalence classes on the 
corporal, buck sergeant ’and authority, private. 

sergeant To establish an 



Theory and Methods of Social Measurement 


481 


(CD), each larger than the step between corporal and buck sergeant 
(BC), were, m these data, incomparable The simplified techniqiie 
used did not yield data which contained this information, hence 

no conclusion can be drawn . 

in the process of constructing an ordered metric sale or 

■authority," we have actually constructed two of them, o"" ^y def 
nition and the other as “perceived ” The question naturally arnes 
as to which one is “better” or "right ’ This gives rise to a basic 
question What is "authority,’ anyway, and, for that J 

what IS an “attribute'’? It will not be profitable to pursue th« s“bj«: 
here, instead it will be sufficient to point out that 
doctrine of operationism plays its role The concept o 
has precisely such meaning as resides in the opera meanings 

observing it To endow either of these scales wit a 
and implications associated with the concept o a 
operationism in reverse and therefore specious a, trussed 

To summarire, the types of scales which have been d. cu« d 
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and the ordered metric In this order they 
more powerful levels of measurement, in the sense 
satisfy successive levels contain more and more m or 


The Interval Scale 
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the Unfolding Technique (6) (discussed in the last two sections), it 
IS possible to take a single induidual subject and determine the 
structure of the attribute a ithority as he perceives it for these 
stimuli It can readily be determined whether his perception of 
these ranks satisfies a simply ordered system and what some of the 
metric relations are 


Let A ~ private 
B = corporal 
C = buck sergeant 
D = master sergeant 


To illustrate the concept of an ordered metric scale, these four 
stimuli were administered by the Method of Similarities to two 
individuals and each individuals structure of the concept of 
lain ° lamed The stimuli were presented to the sub 

no were of each triad which 
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(CD), each larger than the step between corporal and buck sergeant 
(BC), were, in these dita, incomparable. The simplified teclinique 
used did not yield data which contained this information; hence, 

no conclusion can be drawn. , j . , 

In the process of constructing an ordered metric sale for 
•■authority," we have actually constructed two of them, one by defi- 
nition and the other as -perceived." The question naturally arise, 
as to svhich one is “better" or ••right. This gives rise to a fiasic 
question; What is ■■authority." anyway and, for that matter, just 
what is an •■attribute^^? It will not be profitable to pursue this subject 
here; instead it will be sufficient to point out that this is where the 
doctrine of operationism plays its role- -The of authority" 

has precisely such meaning as resides m the operation involved in 
observing it To endow either of these "".‘ho jncaning, 

and implications associated with ‘he concept of authority" 
operationism in reverse and there , 
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arithmetic may be meaningfully performed on the differences be 
tween these numbers 

It will be easiest to illustrate the concept of an interval scale 
by reluming to the ordered metric previously constructed for 
authority and to try to convert it into an interval scale To do 
this, a decision must be made as to what shall constitute a common 
and constant unit of authority for all classes or ranks Suppose one 
decided that command over one private constituted a unit of 
authority, and that a corporal would thus represent twenty units 
of authority A buck sergeant who commanded two corporals each 
cornmandmg twenty men would, on this basis, be regarded as having 
eighty units of authority A master sergeant, commanding three 
uck sergeants and all their men, would then represent 480 units 
of authority (120 privates at one untt each, six corporals at twenty 
units each, and three buck sergeants at eighty units each) 

sented as fono” 


Scale score* 
1 

20 

80 

480 


Rank 
private 
corporal 
buck sergeant 
master sergeant 

may ''be ‘olid" ‘"'"1 

said that rruck sleant 

corporal, a master sergeant Lldoo'"!' ““'''“"'y '^an a 

a buck sergeant, and consequently theTtter"""' 

IS 6^3 times the increment lom a^corn' i "> authority 

A significant aspect of an mte 
associated with the points on the 

linear transformation For examni ’ °"’y "'‘'•’m a 

'’‘umple, any given number can be added 
* This again is an example of how 

measuring it There are other defin^„. '* endowed with meaning by 

result in nonlinear transformations of ihM eould be made which would 
argue that a buck sergeant command! “«ght 

under the corporals and similarly thar corporals and not the men 

buck sergeants The scale scores would then sergeant commands only the 

so measured bore a linear relation to J!! ’ ‘^0 Then if authority 

measured in the text above would hate a rwf variable authority as 

n tnear relation to that variable 
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to all the scale scores and the reUUons between ^ 

preserved. This is called "translation” and 
Lt the origin, the location of the numertcal vato 
trary. Similarly all the scale scores may be „p„-ed This 

number and the relations between intervals ->>>7 
is called "scalar multiplication" and 
the unit of measurement is arbitrary. A linear 
sitting of both a shift in origin and a change m the umt of measure 
ment will also preserve the relations between mterva s. comoare 
This is an important consideration ,f one 
the authority of a sergeant with, for examp e. authority 

loreman of I repair crew. In the ■----^‘'’^^ralgfuhTe 
might be chosen to be ‘‘one repairman. 
comparison then requires that the origin, , . j 

scales and that the unit of measurement be “>entica . This js^the 
reason why one cannot compute the [ people, 

between the mean height and the mean weight of “ ^“P 
As numbers are being used in each apj Jasy to do 

the operations of arithmetic because this is p _ees of the 

in the abstract system of numbers; but the 
operations on the numbers do not necessari y 

real system of objects which ‘he numbers represent For mch a 

of significance to be valid, both scales permits 

same interval scale. Under such conditions the It 

the valid use of most of the tools of "'^‘hematics and stansu^ 
is interesting to note that the product pfLas. 

require that both variables have the same inde. 

urement because this index of relationship i 
pendent linear transformations of the variables. 


T'he Ratio Scale 

To complete this discussion of ‘h' '.“"°'!^^'p''^do scale is an 
jncnt, the ratio scale will be presented y- origin is 

interval scale with the additional cliaracten cjrcumstances, 

an absolute zero and not an arbitrary zero. .,^ 1 .. on the dif- 

»>C operations of arithmetic are perm -Wo not mereh on . 
icrenccs. as svas the case in the mtcrsal sea . i „ on a ratio 

'homselves. The numbers associated with scale sallies 
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scale are then right within scalar multiplication, a consequence 
of the fact that only the unit of measurement is arbitrary Under 
these conditions, it is possible to compute a meaningful ratio of 
two scale values Suppose, for example, in the case of the interval 
scale of authority previously constructed, that zero could be prop- 
erly associated with the scale value of a prisoner of war® as an 
absolute zero of authority Then the scale values can no longer be 
translated but can only be multiplied by a scalar. The buck sergeant 
can then be said to have four times the authority of a corporal, and 
a master sergeant to have 24 times as much authority as a corporal 
These relations would of course, be preserved under scalar multi 
plication 


Measurement vs Scaling 

Because of the considerable increment in power between an 
ordered metric and an interval scale, there is a tendency to distm 
guish between two broad classes of scales The theory of the ordered 
metric and less powerful scales may be referred to as scaling theory, 
^ interval and ratio scales as measurement 

t eory (7) The former may also be thought of as qualitative 
measurement (if this is not a contradiction in terms) and the latter 
as quantitative measurement Because of the composite character of 
the scales with respect to their logical structure, there is also a 
theory t^fer to the entire domain as measurement 

now nTT Pf^^'tous discussion of levels of measurement, it should 
On the broad aspects to measurement 

wuh cerr„ nr " of elements 
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mams of "objects" rn the real world and the.r 

and relations Measurement may then be the 

mapping a real object system into oiie o ^t operations in 

purpose of the mapping being, in p . ^ However, 

the abstract system for operations on J meaninff in 

m order that the operations in the abstract sys e 3 x,ojns on 

terms of the real object system, it is necessary 
which the abstract system has been bnilt be satisfied by the objec 

''"to say that measurement may be ”X"whTt 

object system into an abstract system IS botmme^^^^^ ^ general 

IS sometimes meant by measurement Fr 
sense measurement may mean 

(1) mapping an object XTrimbers m objects, and per 

permitting the assignment of arithmetic to be 

muting at least some of the operations of anthm 

performed on these numbers ^ 

(2) mapping an object system m , nj ratio 

including the ordered metric, interval scaie, 

(3) 'alLralitation to the extent of mapping an object system 

into a partial order or even a nominal seal ^ 

(4) a generalization including ' measurement in 

orders into sets of simple order 

a multidimensional space includes the 

Under this last generalization, measur , ^ be for dua 

enure subject of analysis of data This is as it 'houW be 
>nalys., IS an integral part of the logic of measurement theory 

The l^iicmma of the Social Sctentist 

It IS now evident that the process of system, into 

P^tt of selecting a level of measurement, a *.cteins is a'ail 

which the data are to be mapped A variety o si of 

^blc, \arymg in level from weak to strong *,i,Por> about the 
level of measurement selected constitutes . that 

hehavior in question in that the axioms spcci y ^^j^^^^osliip Tliese 
•te to lioid m the data and the properties 
•looms tlicn become part of the data in llie 
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precedence over the data If there should exist any data incom 
patible with the axioms of the level of measurement selected, 
these data constitute error, by definition 

Let us illustrate this by a hypothetical example of rating 
leadership ability Suppose we have a group of individuals whose 
leadership abilities it is desired to determine, and suppose a group 
of judges who know each of the individuals are asked to rank order 
them in leadership ability We shall require that leadership abilit) 
be measured on an ordinal scale This requires that if A has more 
leadership ability than B, and B has more than C, A must have 
more than C, for all triplets. A, B, C Inasmuch as we required each 
judge to rank order the individuals our method of collecting data 
has imposed a simple order on leadership ability as far as each judge 
IS concerned But vse find that the judges do not all agree on the 
rank order of the individuals m leadership ability Consequent!) 
we must make some further assumptions, usually pertaining to ran 
domness of errors a constant origin, and a constant unit of measure 
ment Such assumptions as these will make possible the combination 
of a number of different rank orders into the single rank order 
which IS required One may then return to the data and specify 
m detail the errors which each judge made It would perhaps 
be interesting to face the individual judges with their alleged 
errors and study their reactions 

The problem implicit m the situation can be called the dilemma 
of the social saentist-which in its simplest form is the problem 
of what shall be called error Almost an>one is willing to sas that 
any gncn set of data contains some error, but just what is to be 
dassiHed as error depends a good deal on the level of measurement 
assumed to hold in the data 


hmJen he chooses 

whT*er = «n>ple order and asking his data 

whether they satisfy a simple order By selecting a strong enough 

un drmemLrr “"vays'succeed m constfuct.ng^a 

midimensional scale of measurement, commonly an interval srale. 
thus requiring a portion of the data to be claL.fied as error 

m =““‘-1 permits the datl 

nmH.3 ““Pleumdimensional soluLn is adequate 

Unidimensionahty. obtained by a method of analysis which g^aran 
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tees it, obviously cannot thereby be shown to 

the behavior in question. This is merely a specia unless 

general principle that no property o£ data can be 

the methods o£ collecting and o£ analyzing the da a p 

live properties to -hibit themselves. The^problm^^ 

scientist, in blunt terms, is whether 

whether he wants to know. . trpnnentlv 

There are several reasons why social scien i 
choose a strong level o£ measurement rather tha 
represent the £u. It is always more p-htab a m use strong scale 
in prelerence to a weak one -'hen botb sat.sly the^^^ 
a more powerful mathematics is available r'jnk.order cor- 

and analysis o£ the data. Compare, j^e powerful 

relation methods appropriate to an ordma ^^thods which 

systems of linear, multiple, and nonlinear “"f 
require an interval scale. There is no wonder that measureme 

by an interval scale has been a major objective. reason 

There is another, and in some ways a curious h«”“hd. m^so ^ 

lor the social scientist to choose a stronger e least a 

than is satisfied by the data-society often requires "h^t oUoast^^^ 
simple order be imposed on an attribute, us, 
esthetic merit of pLtlngs, an ."/J^eVd^ci^ 

problem of choosing a painting to buy. ^ 

he must, in spite of the existence of only p 

simple order over at least a segment ol I • . , ,,(.i,oiog. 

between what appears to be the '"herent nature o gy 

leal attributes as revealed by the data and t 'o “ uj ,1,^ 

society that at least a simple order be impos ' cxplana- 

problem which the social scientist faces. This is 1 1 p jj.jjutinc'' 
tion of why the social scientist must so freqiicn > I 

end. in elTect, be forced to treat his measurement tlicon 
es “right" in spite of his data. „,,nninc a partial 

This situation, the common social nccessi > ^^P^ solution 

order into a simple order or stronger ey's'™. ‘ ..gost" solutions 

««pt by Hat. There may be a number ° '':'^':;";i„'/;„„Tver,ally 
for dirtcrent purposes, but there appears „_.,ial order into a 

"ghf solution. This enforced mapping of => P- 
‘‘ironpcr s\sicm mny be one of the sources of socn 
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The major implication to be drawn from the discussion in this 
section IS that it behooves the social saentist to become fully aware 
of and acquainted with the subject of measurement theory There 
are no pat answers or conventional rules which can be applied 
routinely and in the absence of understanding The “conclusions” 
that are drawn from an analysis of data are highly dependent on 
the level of measurement assumed and imposed on the data, ‘ con 
elusions may be easily confused with postulates or assumptions that 
ha\e been built into the data A decision must be made in terms 
of the objectives of the specific study as well as on the basis of the 
logic of measurement theory 

This section has discussed the general nature of measurement 
theory, the various levels of measurement, and what is involved in 
the process of measuring These considerations, combined with the 
theory of data presented in the next section, will provide a frame 
work for understanding and relating the various methods of collect* 
ing and analyzing data discussed in the last two ’sections 


A THEORY OF DATA 

There is a great variety of techniques and procedures used 
y social scientists for collecting and analyzing data All too fre 
quently, studies which should have mutual implications are not 
comparable because their methodologies are different The pur* 
poses of the theory of data presented in this section are to provide 
a simple framework for organizing and classifying the methods of 
CO ecting an analyzing data and to provide a mathematical basis 
for relating methodologies within classes and between classes 


Genotypic and Phenotypic Levels of Description 

In the theory of data presented here, two levels of description 
will be recopuzed-a genotypic level and a phenotypic level The 
phenotypic level refers to the observed or manifest behavior, the 
genotypic level to an inferred, hypothetical. latent level of behavior 
underlying or generating the phenotypic level 

These two levels o£ description can be illustrated by the per 
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formance of an individual on a mental test. On 
for example, his behavior on each item ”“Sonted ^ ng 

or tvrong) constitutes the manifest behav.or, the f lev h 

On this level, performance is commonly 

of responses-pLes and failures-or by a numer.cal ° 

the nLber of items right. From the -;"f“"Iffgeno’ 

individual and of a number of other mdivi ua s ^ j,as 

more of the ability than individual B. opnotvoic abil- 

implicitly regarded as a function of the individual s 
ity and certain characteristics of the stimulus ‘f” ."ifa 

test, also measuring arithmetic ability, the m iv j j 

quite different scor!, but a set of J^t a monotic, 

dividuals are conventionally expected individuals on 

IE not linear, relationship to the scores o 

the first test. , «„f,rp«ive of two 

In a different context, suppose that t e mo ^ particular 

individuals always dominated. A given in ivi ^ ughave submis- 
amount of aggrLiveness (ge-otypic i-d ^^^t ^beha« sdi 
sively (phenotypic level) m presenc f^^ assumptions 

dominant in the presence of another. formalized and made 
involved in such a line of reasoning have ^^^ling (g). 

a part of a basis for a general theory p J relationships 

The theory of data presented here ^ ^ ^^f.ections, are a 

between methodologies contained m the nex 

consequence of the general theory. In this P made on a 

treatment will be avoided and the presentation svill be made 

verbal level. 


The 


information Contained in Observations 

There are two fundamental aspects to ‘I’'® on dm 

bermiiion of the information contained in a between the 

phenotypic level, and a definition of t "= re ispects provide 

genotypic level and the phenotypic level. observations. 

the basis for making genotypic inferences ro contained in 

Let us consider what the information i Genotypically, 

’a individual’s performance on an aritlimeuc 
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the stimulus ® an arithmetic problem in this case, is regarded as 
requning a certain minimum amount of some ability for it to be 
successfully solved or passed (the ability is spoken of in the singular 
but It may be constituted of more primitive abilities) The individual 
is regarded as possessing a certain amount of this ability His mam 
fest behaMor consists in either passing or failing the problem The 
assumption is made that this implies that the individual has more 
or less, respectively of the ability required ^ The information in 
each element of behavior, a response to an item in the test, is 
whether the individual has more or less of some ability than the 
amount of the ability required by that item 

This is the kind and the amount of information which will be 
assumed to be contained m the behavior of individuals in responding 
to this type of item Exactly what this type of item is will be discussed 
later when these ideas are generalized 

There are other ways of collecting or observing behavior and 
other types of items These procedures may differ both m the kind 
and in the degree of information they contain Consider, for exam 
pie, the manifest behavior of an individual in stating whether or 
not he will endorse certain opinions A statement of opinion may 
be regarded as representing a particular degree of attitude and 
possibly different degrees on different attitudes On each of such 
attitudes, the individual has a specific degree of attitude himself 
at any moment of time The degree of attitude held by an individual 
will be called his ideal for that particular attitude Whether or 
not an individual endorses a given item is then determined by 
whether he regards the item as being ‘ sufficiently close to his ideal 
t e individual is asked to choose the three items he most prefers 
to en will be assumed that he chooses the three items near 


j... “PP”'"' '*>3' ‘he mfomialion contained in such data is 

_ ^ ® hind of information contained m the example 

• Lstd n ‘he .ndividnal 

passed all items requiring less ability than he possessed, in the 

“-ividusn ,u„up 

a itimuhn miijiion For ilmplicii. It „.n tcganltd ai 

TThc hrm, tong dUm.irfW 

other i,ord! the) are noi objeclhe type iieS^ by guemng m 
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case of the attitude items, he ■ endorsed those nearest to him To 
generalize these ideas somewhat, the individuals T ^ 
be regarded as an ' ideal he passed all the Emetic items on one 
side of his Ideal and failed all those on the other srfe ° 

the attitude items, he endorsed items near his ideal regardless 

direction and rejected items further away Uct^rtlv the 

In these two instances, the manifest behavior is abstractly he 

same The stimuli are classified for each individual " 
those he passed or endorsed and those he ai e 
assumptions by means of which genotypic in erence 

ever, are distinct in these two examples ,n the 

With some methods of collecting the information m the 

manifest behavior differs in degree from the a ove omnion m 
the individual may be asked to rank the statenmn 
the order in which he prefers to endorse them On th bans^of ^he 
assumptions already made, it follows that ‘he s i respec 

in order of their increasing distance from his ideal on the res, 

“blus that the information in these data d.fie« injnd 
from the information in data based on w ic ,,nown 

IS actually willing to endorse In any of the items 

whether the individual would be svillmg to 

nr all of them It is also apparent that the in n”™ based 

order data differs only in degree from that contai ^ 

on which three items the individual would prefer ° 
fftrcff Items the individual would choose are on 
postulates, the first three items in the rank or cr 

A and Task B 

The kinds of behavior which In'c o^'eTof the dmoT 

point ha\e one thing in common from the po icspcct to 

They all involve the evaluation of .innih 
JO Ideal In cadi instance both the n.fcjt behavior 

*nve hypothetical genotypic measures am nnf ihccenotypic 

t^miiis inferences lo be made about the relations "P 1^,^^ 

"'^Snuiides of the stimuli to those of the im ,1,^11 

'nciions of this chapter, particularly m die I I'l I 

"bx conditions are necessary to make inferences ahoiii S 
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typic relationships between individuals and the genotypic relation 
ships between stimuli 

For didactic purposes data involving the evaluation of stimuli 
with respect to an ideal will be referred to as data collected by task 
A All the behavior discussed up to this point, then, is task A be 
havior It should perhaps be pointed out that it makes no difference 
whether an attribute is explicit or implicit in the instructions to 
the subject Thus it is still task A whether an individual is asked 
which candidates he prefers with respect to their attitudes toward 
foreign affairs or whether he is merely asked which candidates he 
prefers This does have a bearing on the interpretations given to 
the genotypic inferences but in the formal analysis of data it is 
irrelevant Similarly, it makes no difference in the abstract system 
whether the individual s ideal is his own, someone else s or one 
given to him by the experimenter The abstract characteristic of 
task A is that a stimulus is evaluated with respect to direction or 
distance^ from a point in space called an ideal 

There is another kind of behavior which can be observed and 


with respect to which data can be collected This type of behavior 
is evaluation of stimuli with respect to an attribute and will be 
called task B Illustrations are contained m the evaluations of state 


ments of opinion as to which candidate expresses a more liberal 
attitude or uhich candidate is more pro union or in rating the 
aggressiveness of an individual Whether or not the judge li is an 
ideal of his own on liberalism etc , is regarded as irrelevant 

Data collected under task D may also differ both m kind and 
egrce When an individual ranks a number of individuals as to 
leir ac ministrative capacities, there is no information in these 
data as to ^^hethe^ or not any of the individuals are good admin.stra 
tors as would be implied in the case of rating them This is an 
3 d'ff^rence m ktnd of information An example of a 
information is picking the three best admin 


uotktd om for dijomion in Ihi. chiDlw 1^,^ f 

alltmolivo derm,. ions of diilance do on ' ‘""■I'memional caie Ihese 



Theory and Methods of Social Meosurement 


493 


Relative and Irrelative Behavior 

In this discussion of the theory ot data, an explicit d.chmomy 
has been made betiveen task A and task B with respect » /he 

o£ the information contained in the data Certain ot er 

have been made implicitly One of these may be re erre o 

and irrelative behavior In relative behavior, the data -■'e based o" 
the relationships between two or more stimuli or examp , 

ment as to which of two candidates an individual P™" 

(task A), or which of the candidates is more pro union ('^k ^ 
the individual rank ordered his preference lor all the and ^ates^ 
It would still be relative behavior, task A, and the m o 



“>e data would differ only in degree from ‘h® s"™Iarly 
fent on one pair Obviously this can be exten (. mvolve a 

In iirelative behavior; the f arulimeuc test 

**^gle stimulus at a time This is illustrate y expressing 

referred to by rating individuals on a rating sea . interest 

ones hkes and dislikes for each of a number oE item 

The two dichotomies developed so Ear, ‘^sk® ^ ^ 

h'e and irrelatue behavior, may be put , discussion, the 

table (Fig 1) in which for simplicity of disci 
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quadrants have been numbered It will be observed that Quadrant 
II has been further dichotomized on the basis of the kind of stimuli 
monotone or nonmonotone This distinction is illustrated by the 
different assumptions involved in making genotypic inferences in 
the passing or failing of an arithmetic item and m the case of 
whether or not a statement of opinion is endorsed In the arithmetic 
Items the individual s manifest behavior is pass for all items on 
one side of him and fail for all items on the other side This type 
of Item IS called monotone Nonmonotone items are exemplified by 
statements of opinion in which the individual endorses those items 


nearest him in a segment of the space surrounding him and rejects 
all Items beyond 

In the case of a single underlying latent attribute the items 
an individual rejects may consist of two subsets one of which con 
tains those that are too extreme in one direction and the other those 
that are too extreme in the other direction Consider, as an example 
of a nonraonotone item the following statement of opinion \Ve 
should make the loan to Britain if we are sure they will pay it 
back On a hypothetical continuum from pro to anti British 
individuals in the neighborhood of the middle of this continuum 
would presumably endorse this statement But the individuals who 
refuse to endorse this statement may be at opposite poles of the 
continuum The very pro British may say no because they ivant 
m nia e a loan to Britain without any conditions and the very anti 
ritis say no because they do not want to make a loan to 
Britain under any conditions Hence two very different kinds of 
S’.tn’’ ' Klentically (phenotypically) m this 

a ion Similarly those individuals in the middle of the con 

""ght be expected not to 

iviZal TiTJ'” ”™'B"''*«atements Hence certain in 

d ltmrt Ivlff r P’'™°'yP'“»y ‘he same to two genotypically 

distinctly different kinds of stimuli ^ 

tone 9 items^arp'^rr of these two types of items mono 

fortes If LmffiT ^ -ip-ng 

gories of manifest behavior into penntvmr- » 1.1 

__ j S^ttotypic categories is possible 

and nonmonotone Items are those fftT- ° ^ 

ts necessary “> "““-y "tapping 

-A type Of Item called cumolauve « a special case of a monolone item 
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Distmclion Between Methods of Collecting and Analyzing Data 

We now have a framework withm which methods of 
and analyzing data may be discussed Within a given quadmm 
(Fig 1), the kind of information contained in the 
and data collected by various methods differ ^ 

of information they contain The placement of a set » data n * 
fourfold table will be seen to be. within certain 
that IS made by the eitpenmenter when he chooses a particular 

-.u a.,™- 

they contain, but the method of analysis , analyzing 

and this IS what situates the data m the table 

data have been devised historically, ud hoe. for each of the ^mnu 
Some of these methods seek only to conso i ™ inferences 
phenotypic information, others seek to make ge yp 
from the phenotypic information aumverv of 

TheWthod of analysis selected 'he mrpemes 

the properties of the information or may a so efi 
In the latter case, the experimenter ts concerned only tv.th the „ r 
telations This « precisely the -locus of the dilemma of the 
tcientist referred to in the previous ^ecuon ^ 

The distinction between methods of _,i.„tanding of the 
ods of analyzing data is imperative for n" undemtandmg^^^^^^ 

telationship between the inferences ‘'■’n"'" hom understand 
The relatio'nships among the quadrants -e d'« 

’ng of the distinction between collecting collecting data 

3lso to the relationships among different met i 

3nd different methods of anal>zing data ..j. 

Irrelative behavior (represented by se as 

"> the abstract, the response of an indivi ^ ^ 

ronirastcd with relative behavior (Quadrants . ' clearly 

>h^ response of an individual is a Stimuli 

•rrelative behavior is represented by the ■ c undine data in 
"1 its broadest sense, and all the melho s , collectively 

Quadrants I and IV, relative behavior, can be referred to collec 

” ’h' 'Icthod of Choice between monotone 

In Quadrant III a distinction is possible between 
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and nonmonotone stimuli, but since methods of collecting data in 
this quadrant invariably use monotone stimuli, nonmonotone 
stimuli have been neglected m this presentation It can be shown 
that Quadrant III is formally indistinguishable from Quadrant 
Ila— the distinction exists only in the frame of mind of the expen 
menter Furthermore, it can be shown that the Method of Single 
Stimuli as a whole, both Quadrants II and III, is a special case of 
Quadrant I 

The distinction between Quadrants I and IV is a real one 
although there are data collected by certain methods which may be 
placed in either quadrant depending upon the objectives and frame 
of mind of the experimenter These distinctions and interrelations 
will be brought out in more detail in the next two sections 


METHODS OF COLLECTING DATA 

In this section a number of methods of collecting data will be 
discussed and some of their interrelations pointed out A potential 
source of confusion resides m the fact that the names for some 
methods imply both a method of collecting and a method of ana 
lyzing data-ff g , the Method of Successive Intervals (28) Throughout 
t is section the mention of any method will have reference only to 
Its use as a method of collecting data 

A general system for structuring or organizing methods of 
collecting data in Quadrant I will be developed on the basis of the 
amount of information each contains The relation of these methods 
when used to observe behavior m Quadrant IV will be pointed out 
Tl|e seme .n wh.ch .he Method oF Single St.muh c£ Quadrants II 
and in ,s a special case ot Quadrant I w.ll then be discussed 


(Quadrant I 

la, n ‘5“^ ‘nIorn.at.on contamed in methods o£ collecting 

data tn Quadrant I. examples ...11 be g.ven tn wh.ch un.d.mens.on 

'“nmples w.ll be used partly because 
the untdtmenstonal case has been more completely worked out and 
also because ol the preoccnpa.ton ol most social sc.ent.sts w.lh un. 
dimensional representations of data 
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Let us suppose throughout the following discussion that we 
have hielinuT": C, D, and £, on sonre latent attribute and 

that the Ideals of individuals are distributed over thts same latent 
attribute with fixed metric relations For concreteness S 
that the five stimuli are statements of op.n.on are 

degrees of attitude toward x, and the ideals of the -"^mduals^are 
the hypothetical statements of opinion which each wo ^ ^ 

endoBe above all others This situation 
are unrealistic constraints, but the simplicity o ^ 
very desirable for didactic purposes These constraints are com 

pletely relaxed in the more general treatment „hirh a fre 

The situation may he illustrated by which a fre 

quency distribution of the ideals of the individuals over 



Tie 2 \n example of a distribution of people and stimuli on a 
Joint scale 


continuum is indicated Such a scale may be j „]5 on it 

or ]omt d.stnbut.on, having both st.mul. and md.v.du^^c^ 
Suppose that one collects data by as '"S basts 

nidicate the two statements he most prefers j_„e a stimulus 

“f the postulate that an individual will prefer to , under 

nearer h.s .deal than one farther asvay, the response patterns 
ihese conditions would be the following 


AB, BC, CD, DE 


It 15 evident that all individuals to the 
^iween stimuli A and C would give the respon 
dmtluals to the right of the midpoint dC and 
^‘dpoint BD %vould ha>e the response BC Tbe 
^ponses to midpoints is illustrated m Figure . indicated b) 

^es o[ the regions associated with a respons a 

Ileal lines sectioning the scale The respons 
^*01 or segment of the scale is also indicate 
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AC BD CE 


, AB , 

BC 

CD 

DE 

"T 1 

A B 

1 

c 


D E 


Fic 3 Relation of response patterns and scale values undei 
PicV 2 

Thismethodof collecting data called Pick 2 in general yields 
n i categories of individuals where n is the number of stimuli and 
the boundaries of regions are given by the midpoints of alternate 
stimuli 

If instead of Pick 2 the individuals had been instructed 
to Pick 3 the response patterns would have been 

ABC BCD CDE 

and their relation to the scale would be as illustrated in Figure 4 


AO BE 


.ABC 

_^BCD 

CDE 

A 

1 

6 1 

c 

r' 1 

1 D 

1 ■ 

E 


Relation of response patterns and scale values under 
Pick 5 


The number of categories of individuals would have been three or 
in general n 2 This procedure Pick k may be continued for 
higher values of A up to n 1 For Pick n 1 only two patterns of 
response would be obtained One pattern would be that of individ 
uals to the left of the midpoint between the first and last stimulus 
the other pattern would be that of individuals to the right of 
this midpoint This might be more immediately obvious if one 
reco^izes that from a formal point of view Pick n 1 is the same 
as Reject 1 In the unidimensional case there are only two stimuli 
which may be rejected the two end ones and the choice between 
these tuo IS dependent on their midpoint m relation to the judges 
idea! In the present example the results would be as indicated in 
Figure 5 
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AE 


ABCD 


BCDE 


A B Cl D E 

Flc. 5. Relation o[ response patterns and scale salucs 
"Pick n-I" or "Reject 1." 

For each of these methods of collecting data Q"' I; 
from "Pick 2” to 'Tick n-l.“ there corresponds a of analyj.^ 

which will reveal certain characteristics of t This 

underlying the phenotypic behavior— the sta e p ^ j 

method of analyst, called Parallelogram Analysis, is discussed 

next section. . „ extended to 

If this method of collecting data. P>c . endorse, 

^ = 1. where each individual indicates the one i the 

there is no unique solution to the genotypic stru Method 

preferences (see p. 513). "Pick 1" is 'P^’^'Tnde Stimuli. This 
of Choice which corresponds to the ar, because the 

u not so severe a criticism as it may at PP 
Method of Single Stimuli is oiUinarily use o y 
a priori ordering of the alternatives to an item a _ become 

3sl;ed to provide it. Methods of analyzing sue number of 

concerned only with the interrelationships bett ^^^ j., 

such scales as the above, one for each item. Qpjdrant II. 

''ill be studied as a special case in its own ng i . rpuirn to “Pick 
Omitting "Pick 1,” then, for the time being, let us return 

“ 3nd continue with relative behavior. indicate their 

If our subjects are instructed ^ "O^er ^ the response pattern 
first and second choices) instead of P*cl^ • ,,«rn« j-lli and 

■IB obtained in "Pick 2" becomes two response Pn respectively, 

^ad these are given by individuals to the Ic t an ^ b^. 

die midpoint Ali. Similarly, each of the midpoints 

'omes two patterns under “Order 2" instriic io ' * to the 

adjacent stimuli on the genotypic scale . r jjjj, regions 

*’"d[)oinis of alternate stimuli to form the bount ‘ illnstratcd 

*nociatcd with each phenotypic response pattern. become 

•" fi^re G. The number of categories of mdiv.duah 
^'Sht instead of four, or, in general, 2 (n-l). 



Fic 6 Relationship of response patterns and scale \alues under 
Order 2 


It IS now apparent that for n stimuli presented at a time, the 
methods of collecting data from Pick n 1 to Order 2' (omitting 
Pick 1 ) contain a monotonically increasing amount of information 
in the sense of the number of categories the individuals are classified 
into on the basis of distinguishable response patterns The method 
of analysis of information (the Parallelogram Technique) leading to 
genotypic inferences is the same for all and will be discussed m the 
next section 


It IS obvious that this series of methods of collecting data may 
be extended to ‘ Order 3, ’ Order 4, down to Order n 1 The 
latter corresponds to the Method of Rank Order These methods 
of collecting data may be distinguished from the preceding methods 
or although both tjpes are logically contiguous, methods xvhere 
a number of items are ordered yield a new class of information- 
information about metric relations 


A detailed presentation of the theory and technique for extract 
mg information about metric relations in rank order data is con 
tamed in the literature (6) Here the case of Orders mil be 
Illustrated Keeping m mind Figure 2 and the psychological postu 
tales n Inch hare been made, consider rshat the data would be like 
<“ kft of the midpoint AB would 
yield ABC^ their phenotypic behavior Individuals to the right 
o thts m.dpo.nt AB but to the let, of the m.dpo.n, AC would act 
like phenotypically and would yield BAC Crossing the midpoint 
AC reverses the order of these two sttmul, ,n the phenotypic be 
havtm. and the next order.ng „„„,d be BCA If th.s procesl ts con 
t uned for the s.tuat.on given rn Figure 2 ihe complete results are 
those given in Figure 7 ^ 


In comparison with • Order 2." where .here were eight classes 
of tndiv .duals or, tn general, 2(n 1), here there are ten or. m general, 
Sn 5 Hut there ts. tn add, non. a new k.nd of mformat.on .n these 
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Fic. 7. Relation o! response patterns and scale values tmde 
“Order 3 ” 


lista-information about metric relations. Const er 
points BC and AD. In this instance the midpoint p 
iitcause the interval between the two stimuli -j-jic pheno- 

than the interval between the two stimuli C an ' , j (Rg two 
>ipic behavior associated with the interval boun e 

midpoints BC and /4D is CB/I, and this phenotyp .-y; ^ 

■iie genotypic implication that AB < CD. If. fot the 

•fhence^he AD midpoint had preceded i,„„ 

Pfitnotypic behavior in the region they bounded w 

®CD instead of CBd. . ,i,at BE pte- 

Similarly, the phenotypic behavior CDE ‘™P ‘ ^ ]n 

“des CD, which has the genotypic '"’P'ff with respect 
E^neral metric information from "Order 3 is o -jg between 

the relative magnitude of alternate single m 

'Order k" is employed as a method of col ’ ^ ^alue 
f ormation contained in the data includes that ^ 4. 

and additional metric information as k incr . jg^Qfsums 
'formation increases to include the intervals. The 

® jacent intervals compared with sums of a ^^f^..t-)rdcr n-l," the 
^ nd containing the most metric information is 
‘ 5 Rank Order. . ;„.nlvinf: “Order 


llic llluat invve**' 

Of Rank Order. . involving “Order 

, or the sequence of methods of collecting analysis for 

, t n.l, there is n corresponding "-f hod of an J 

the genotvoic inferences coniaine The r 


n-l, there is a corresponding data. This 

. o ‘“c genotypic inferences coniaine » raiih 

of analysis is called the Unfolding ^ simply 

the stimuli for an individual will be ca . ^ic indh'i^f*'^^ * 

may be regarded as the J scale fol Tcchniqnt^ to 

. this which gives the name the hfr* ° generate 

of tots of I scales Jphenotypic hehav.or) 


AlBll Cl lIDIt 

Fig 6 Relationship of response patterns and scale salues under 
Order 2 

It IS now apparent that for n stimuli presented at a time the 
methods of collecting data from Pick n 1 to Order 2 (omitting 
Pick I ) contain a monotonically increasing amount of information 
in the sense of the number of categories the individuals are classified 
into on the basis of distinguishable response patterns The method 
of analysis of information (the Parallelogram Technique) leading to 
genotypic inferences is the same for all and will be discussed in the 
next section 

It IS obvious that this senes of methods of collecting data may 
be extended to Order 3 Order 4 down to Order n 1 The 
latter corresponds to the Method of Rank Order These methods 
of collecting data may be distinguished from the preceding methods 
for although both types are logically contiguous methods is here 
a number of items are ordered yield a new class of information- 
information about metric relations 

A detailed presentation of the theory and technique for extract 
mg information about metric relations in rank order data is con 
tamed m the literature (6) Here the case of Order 3 will be 
strate Keeping m mind Figure 2 and the psychological postu 
lates uhich have been made consider what the data would be like 
°^i,i individuals to the left of the midpoint AB would 

yield ABC as their phenotypic behavior Individuals to the right 
of this midpoint AB but to the left of the midpoint AC would act 
alike phenotypically and would yield BAG Crossing the midpoint 
AC reserses t^he order of these two stimuli i„ the phenotypic be 
hasior and the next ordering would be If this process is con 

1 ue or t e situation given in Figure 2 the complete results are 
those given in Figure 7 ^ 

In companson with Order 2 where there were e.ght classes 
of.ndis.dualsor m general 2(„ I) here there are ten or ,n general 
^n5 But there ts m addition a new kind of tnformat.on ,n these 
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data— information about metric relations Consider the two mid 
points BC and AD In this instance the midpoint BC precedes AD 
because the interval beuveen the two stimuli A and B AB is less 
than the interval between the two stimuli C and D, CD The pheno 
typic behavior associated with the interval bounded by the two 
midpoints BC and AD is CBA, and this phenotypic behavior has 
the genotypic implication that AB < CD If, for example AB > CD 
and hence the AD midpoint had preceded the BC midpoint the 
phenotypic behavior in the region they bounded would liave been 
BCD instead of CBA 

Similarly, the phenotypic behavior CDB implies that B£ pre 
cedes CD, which has the genotypic implication that > ffE In 
general metric information from * Order S’‘ is obtained with respect 
to the relative magnitude of alternate single intervals between 
stimuli 

If ‘ Order k' is employed as a method of collecting data the 
information contained in the data includes that for any lower value 
of k and additional metric information as k increases Por A ^ 4, 
the information increases to include the relative magnitudes of sums 
of adjacent intervals compared with sums of adjacent intervals The 
method containing the most metric information is ' Order n 1, the 
Method of Rank Order 

For the sequence of methods of collecting data involving Order 
A, 3 ^ A ^ w 1, there is a corresponding method of analysis for 
obtaining the genotypic inferences contained in the data This 
method of analysis is called the Unfolding Technique The rank 
order of the stimuli for an individual will be called a simply ordered 
I scale and may be regarded as the J scale folded at the individual s 
ideal It IS this which gives the name the Unfolding Technique to 
the analysis of sets of I 5cales_(phenotypic behavior) to generate 
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a J scale (a genotypic inference) The Method of Parallelogram Anal 
ysis Will be seen to be a speaal case of the Unfolding Technique 
The next two methods ol collecting data to be considered are 
the Method of Paired Comparisons and the Method of Triads The 
Method of Paired Comparisons for Quadrant I behavior constitutes 
the presentation of all possible pairs of stimuli with the instruction 
to the individual to indicate a preference within each pair The 
Method of Triads for Quadrant I behavior constitutes the presenta 
tion of all possible triplets of stimuli with the instruction to indicate 
which IS most preferred and which is least preferred A further 
generalization is clearly possible but would be beyond the scope 
of this chapter 


In the Method of Triads, each pair of stimuli is judged n 2 
times, where n is the number of stimuli Each pair of stimuli is a 
constituent of a triad with each of the remaining stimuli m turn, 
and when the individual says of the stimuli A, B, and C that he 
prefers A most and C least, he is placing them m a rank order of 
pre erence A B, C This rank order of preference is equivalent to 
three transitive paired comparisons A > B, B > C, A > C, where 
. ^ preferred It is important to note that this transitivity 

has been imposed by the method used m collecting the data The 
ju gments of an individual on a triad may be decomposed into 
I tree paired comparisons which have two degrees of freedom One 
'k* of telling of course, which two paired comparisons are 

the independent ones 


of *t.^^^** paired comparison judgment between a given pair 
nprm !i!* ® ^ times, it IS apparent that the Method of Triads 

S ^ ^ P=‘red comparison judgment to be 

Commri.nn^ Context of a third stimulus The Method of Paired 
onlvonre paired comparison judgment is made 

LsuTn. rnn^ ***« cousisfency of a^dgment but. 

of the naired permits testing the transitivity 

now /pparent that J 

on.he.n.p,.edpa.redcomprn»n*r.: 

have methods of collecting data, we 

ture ts the method among those dtscussed here that contains the 
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most information about the behavior being studied The other 
methods of collecting data are listed m order of decreasing informa 
lion For convenience of disaissiori, jt is desirable to gi\e a name to 
the implied attribute underlying this power structure The term 
“seardiingness” is suggested If ne measure a unidimensional attn 
bute, the methods listed m Table I, then, go from tiie most searching, 
at the top, to the least searching, at the bottom 

There are two other data collection methods m common use 
which should be included here These are the Method of Equal 
Appearing Intervals (17) and the Method of Successive Intervals 
They may both be regarded as adaptations of the Method of Rank 
Order in which ties m rank are permitted In the Method of Equal 
Appearing Intervals, the subject is instructed to place the stimuli 
in a given number of ranks, equally spaced psychologically, in the 
Method of Successive Intervals, he is merely given the number of 
ranks (piles) with no constraint on their ‘spacing These methods 
although less searching than the Method of Rank Order, are not 
directly comparable to any of the other ’ Order k methods This 
might be more evident if one recognizes that Order k" for ft < n I 
yields a segment of one end of an I scale The Method of Equal 
Appearing Intervals and the Method of Successive Intervals yield 
the entire I scale collapsed into steps which are or are not, respec 
lively, assumed to be equal psychologically 

A more thorough development of the power structure on raeth 
ods of collecting data, including methods not covered here, reveals 
that they are partially ordered with respect to the amount of 
information they contain about the behavior m question 

There are some general implications and interpretations which 
follow from the power structure on methods of collecting data con 
sidered from the general point of view of measurement theory A 
selection of a method of collecting data and a method of analysis 
in any particular instance is dependent upon a resolution of the 
dilemma that the social scientist faces Having resoh ed the dilemma 
to suit his purposes, he can select a method of collecting data and 
a method of analyzing it which will, for example, guarantee that he 
will end up with a umdimensionai scale, or, if his objective is dif 
ferent, that will provide a test of dimensionality 

One of the basic issues in the interpretation of data is illustrated 
by the example of an individual who sajs, in three successive judg 
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ments, that he prefers A io B, B to C, and C to A One does not 
know whether the individual s judgments are inconsistent or whether 
they are intransitive With a technique as searching as the Method 
of Triads, it appears possible to make a distinction between behavior 
which may be classified as error and behavior which requires ex 
planation By behavior which may be classified as error is meant 
behavior of a single individual which is random over replications 
of the stimulus situation If the Method of Triads revealed that the 
individual was consistent and intransitive, it is incumbent upon 
the experimenter to accept this as experimental fact, regardless of 
the behavior of this individual on other stimuli or the behavior 
of other individuals on these stimuli All conventional methods of 


measurement or scaling which classify intransitive judgments or 
other portions of data as error make one or more of the following 
assumptions (1) that different individuals are replications of one 
another for the same stimulus situation. (2) that different stimuli 
are replications of one another for a given individual (3) that a 
theory (level of measurement) is valid in spite of the data It is now 
evident that these assumptions are neither necessary nor desirable 
un ess t e experimenter has resolved his dilemma by deciding to 
construct a unidititenstonal scale tn spite of the data 
, “ data collection method less searching than the Method 

ria s ts emp oyed, a distinction between inconsistency and in 
transitivity in an md.v.duars judgments ts no longer possible unless 
PairM r assumptions above are made The Method of 

tied Comparisons imposes consistency (reliability) on the judg 

irscs thn", of Rank Order further 

ZparUn nd paired 

be expressed a '"dividual are transitive, the data may 

d ta aTcoltr ed'r t,°1" ™ '"formatton But if thl 

P “ed com™ ‘■’n '^""""v"y of the 

of obsen Zn r, been imposed on the behavior by the method 

bLn .rrutle or 00 .“ "" 


As we move down the searchmgness scale, there is a senes of 
success.vely decreasing numbers „t elements tn the rank Ler Where 
the number of st.mul. ts n, the individual ,7 a v . 

order of preference only for the H^rr' <\'’‘Xe 
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methods impose all the properties that the Method of Rank Order 
does but, because the rank order is incomplete, the missing segment 
IS in effect determined for eacli individual by the judgments of the 
other individuals In other words, since we have no information 
on the end segment of an individual's rank order, it immediately 
IS compatible or in agreement with any obtained data Thus, in all 
methods of collecting and analyzing data, that information in data 
not collected is always regarded as compatible with the information 
that was obtained 

If a simply ordered scale of stimuli and judges is desired, data 
with too much information in it may contain information incom 
patible with the desire By the use of a method of collecting data 
which will provide less information, such as a ' Pick k method 
instead of an “Order k," a simple order of all the stimuli may be 
constructed which is inferred to hold for all This illustrates a 
general interpretation that may be given the searchingness structure 
on methods of collecting data In one sense the searchingness struc 
ture may be regarded as a set of criteria for unidimensionality of 
behavior, the less searching being the weakest criteria and the most 
searching the most rigorous Behavior which under a given criterion 
appears unidimensional will so appear for all weaker methods but 
may or may not satisfy unidimensionality under the stronger criteria 

There is another implication of this power structure on methods 
of collecting data~the fundamental principle that social science data 
are worth no more than the ‘ effort expended by the judges in mak 
tng their evaluations This is illustrated throughout the power 
structure It is easier for a judge, for example to “Pick 2 than to 
Order 2 ’’ The latter contains the ' Pick 2 ' information and more 
The pnnaple is again illustrated by the relationship of the Method 
of Tnads to the Method of Paired Comparisons 

The Method of Tnads for relatue behavior, task A, is at the top 
of the pouer structure for all the methods considered here and is 
minimal in the properties it imposes on the data In fact this method 
of collecting data permits “almost anything to happen,” and the 
mherent variability and other characteristics of the behavior are 
permitted to reveal themselves The Method of Triads requires so 
much effort on the part of the judges, however, that it is imprac 
bcal for a large number of stimuli, and this is probably the pnmarv 
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reason for its being used so little For the intensive investigation of 
Quadrant I behavior, over a moderate number of stimuli, it is the 
best of all methods presented here 

Because of the large amount of information in data collected 
by the Method of Triads, a judiciously selected portion of triads 
can be substituted for the Method of Paired Comparison when the 
latter method appears too formidable for the judges This results 
from the fact that each triad may be converted into three paired 
comparisons with two degrees of freedom One of the objections to 
the Method of Paired Comparisons is that it is tedious for an in 
dividual over a large number of stimuli For n = 20, the number 
of paired comparisons required of an individual is 190 The full 
Method of Triads for 20 stimuli would require the individual to 
make judgments in 1140 triads, and each triad is the equivalent of 
three paired comparisons with two degrees of freedom It is possible, 
however, to select 63 of these IHO triads which, with one additional 
paired comparison, would be equivalent to the 190 paired compari 
sons, but the number of degrees of freedom would be 127 instead of 
190 If the 190 degrees of freedom were required, it would take 95 
triads which would be decomposable into 285 paired comparisons, so 
that some of ihe 190 different paired comparisons would be repeated 
This presentation of methods of collecting data has by no means 
exhausted the variety Enough has been presented to permit the 
construction of a simple power structure, one of the directions for 
further generalization has been pointed out, some of the implica 
tions of the power structure have been indicated 


Quadrant IV 

“P to tim point has been entirely in the con 
Prefer' '>>= mdiv.duals indicate comparative 

“f potentially at least, 

avadable for nse m Quadrant IV also, vnth an appropriate change 

o'! the comn T , bthtivior of individuals consists 

of the cotnparattve evaluat.on of aumuh with respect to an attribute 
Thus, for Pick *. an individual would be asked to 'Pick the k 
most aggressive children in this group 

The infomation m such judgments is not which of the stimuli 
IS nearer an .deal of the judge on some underlying attribute, as in 
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Quadrant I, but which 

There is here, m princip . ^ judgment 

ideal which the judge may Y , ^ prefer candidate 

Let this be made very clear one individual^^^ J P^^^ 

.1 to iJ and another one B to A. for the 

has different ideals, for one ju g , whether A 

other, d IS nearer or speaker, or 

or B was more conservative ‘ ^r the hypo 

more pro union, etc , it is no attributes 

thetical preferences of the two judges on any 

have any relationship “ two individuals one m 

In concrete terms it is assume training will not 

favor of and the other against following statements is 

for that reason, disagree as to which of 'be folio g 

more favorable to universal -"‘''""y 
age of 18 should take one year of mill ry g 

(2) All men at the age of 18 snoiiia u 6 

of military training ,„n„rtina data by the methods 

The immediate objective of “M .hat the 

of Quadrant IV is to study stimuli on the 

data are analyzed to ob'®'" ““ , interval scale With such 

attributes, commonly m 'be t°c structure would not 

an objective most of the met o sufficient spread on the 

be useful because the “'™'''‘p®'!“.’'ji,g,.I,oices of most individuals 
relevant attribute that under i rhosen could not be scaled 

would be the same and 'be powerful methods 

Hence the methods generally u romparative judgments in 

which require each individual nods include the methods of 

many reg'lons of the 
Equal Appearing Intervals ouc 

Paired Comparisons (33) and , , 5 ,^ 5 instructions can 

Data collected by these .00*0* und^a^k 

not be analyzed by the Unfo i g respect to an ideal The 

do not involve evaluation of stimu i »» .ug Law of Compara 

method of analysis used on such data is “*b= 
live Judgment (33) or an ''PP''“f’T ' j^sis on paired comparison 

of course use Ihis latter method , 

ludgments collected under task relationship to ihe results 

analysis will bear a distinct algebraic 
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obtained from applying the Unfolding Technique to the same data 
The Unfolding Technique yields a genotypic analysis of preferences 
the Law of Comparative Judgment, or its modifications, yields a 
phenotypic analysis These two analyses, then, have a predictable 
relation in this general theory This is an example of a single set of 
data (collected by the Method of Paired Comparison, task A) which, 
depending upon the attitude or objective of the experimenter, may 
be located by his method of analysis in Quadrant I or Quadrant IV 
To extend the Unfolding Technique to the analysis of Quadrant 
IV data a somewhat different method of collecting data must be 
employed This method may be called the Method of Similarities In 
this method the individual is presented with the stimuli three at i 
time as m the Method of Triads The instructions, however, are to 
judge which pair of the three is most alike and which pair is least 
alike The instructions may or may not indicate an explicit attribute 
— i e , the instructions may be to judge which pair of a triad of 
cultures is more alike in their ethics, or the instructions may be 
merely to judge which pair is more alike In either case the analysis 
of the data follows the same procedure TVith the attribute explicit, 
however, the Method of Similarities will yield the latent structure 
underlying the stimuli for this attribute as perceived by an individ 
ual With no explicit attribute, the method will yield the latent 
structure underlying the similarities and differences of stimuli as 
perceived by an individual 


^is technique was used by Richardson as a method of collect 
mg data m his two dimensional study of color (27) The method of 
analysis, however, was a carryover of Thurstone’s Law of Com 
parative Judgment The method suggested here for analyzing these 
data IS quite different-namely, the Unfolding Technique This 
difference m method of analysis is a direct consequence of the 
definition of the information contained in the data To apply the 
Unfolding Technique to such data, the information they contain 
IS obtained as follows The responses of the individual to each 
such mad may be converted into three paired comparisons, each an 
element of a different I scale For example, suppose an individual 


B 




responded to the triad 


with the statements that B and C 



Theory and Methods of Soc.ol Measurement 


511 


were most alike and B and ° B. the foregoing 

asked whether stimulus ^ have said C In effect, 

judgments imply that the in ivi ^ j ,(jeal in the 

ihen. the individual was 

stimulus space and was saying P judg 

IS preferred to D " This V’ “ 1 -Iividual standing at the 
ments that make up an I scale tor me 

position of stimulus B rpsnonses to this one triad 

In exactly the same 7""- ^ .n^^dual s I scale when 
provide one of the judgm u„menls for his I scale when he 

L IS at stimulus C and one of the ^stimulus C, B is 

IS at stimulus D Tlius, from t e p ^ ^ preferred to B 

preferred to D, and from the poi triads, an I scale, 

From the individual’s responses to (or the individual 

not necessarily transitive, may be constructed 

standing at each stimulus position I,ostility relations among 

Klingberg (19), m his “7^ 1) states m order 

sovereign states had each su je state Each state in 

of decreasing Tht technique necessarily yields tran 

turn was used as the Standard r c *nilarities may yield mtransi 

sitive I scales, whereas the Method of Similarities may y 

tive ones „„,i,nds mav be analyzed by the 

Data from either of the a ove structure underlying the 

Unfolding Technique to smdy ^ apparent, then, that 

stimulus domain for a single in i tv contains informa 

such a method of collecting data m Q Quadrant I, and the 

tion which permits it to be ^ I may be applied to 

methods of analysis appropriate ^ j^^ngness the Method of 
these data In the P°7 '““"Method of Paired Comparisons and 
Similarities corresponds to the uonV Order 

Klingberg s technique to the Method of Rank Orde 

Quadrants II and III “Pick 1 

In the power structure of “al^as^ of^the Method of 

as a method was mentioned as a sp « i- Stimuli This will 
Choice, corresponding to the method of Single 

now be discussed in more detail hehaMor, monotone 

Consider an eKample of Quadrant HI 
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Item a judge evaluating a subject by means of a rating scale 
From the point of view of the information in the response, he may 
be said to be taking as his ideal his notion of the subject’s ideal, 
and from that position he is picking one of the alternatives on the 
rating scale as being the stimulus nearest this ideal When the ideal 
which the judge takes is actually his own ideal, task A explicitly, 
the behavior is classifiable as Quadrant IIu But in terms of the 
information in each response, there is no difference between Quad 
rant II and Quadrant III, they are both “Pick 1" among a set of 
alternatives being evaluated with respect to an ideal 

Thus, monotone questionnaire or attitude items, typically of 
the Likert type (21) with alternatives running from “strongly agree” 
to strongly disagree,” are simply rating scales where the data are 
collected by Pick 1 ” This type of data is tolerated even though 
It contains no information on the order of the alternatives because 
the experimenter has an a pnori order which he regards as uni 
versally acceptable This constitutes a generalization of the concept 
of right answer " As far as the data are concerned there are nPn 
possible simple orders, as will be seen in the next section, but this 
does "“' matter to the experimenter because he knows which one 
18 rig t t e data were collected by any of the other methods in 
the searchingness structure, tests for simple order of the alternatives 
an or common metric relations among the alternatives would be 


th^ J*"! of Single Stimuli is from 

reurpZ, f ‘"="* Eoch 'tern i, regarded as 

thu attribm/" “Iternatives are the stimuli on 

?nlv tw?al ® ''■‘‘8“ "picks I When there are 

See Z,^!r endorse, pass fail. 

Method ofs “ another way of looking at the 

Items taken wMe Tn’Xl‘'th""‘ 

MmSrofwtrt --- ”rs:::„:;.,Ten ^ 
s“ s In^r Lrod'S"* "■ 

r 
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thereby implied. The methods of Te" 

include the systems of ^uttman ( 6) Lar^ 

Theory (15). Their relations and distinctions will 8 

the next section. 


METHODS OF ANALYZING DATA 


Paralleling the f f th'f methods of 

contained in the data. „„„r„„riate to Quadrant I 

The methods of Technique and the 

behavior, which include the Para e gr ^ followed 

Unfolding Technique, will be d-s^s^d include 

by the methods appropriate to Quad derivatives and the 

the Law of Comparative Judgnjent and 

extension oE the Unfolding Techniq methods appropriate 

ities; finally the discussion will consider L,,ars. 

to Quadram II and III, which and Test 

fern system of theories, within. which Gunmans theory 

Theory are special cases. 


dnalyjj 5 of Quadrant I Data ^ 

..IT Paralleloeram Analysis is spe- 
THE PARALLELOGRAM TECHNIQUE. methods 

cifically designed for analysis of data ro ^ (j items as 

from •■Pick n-1" to "Pick 2.” A maw.x - “"^^l^ses an item, 
columns and individuals as rows. If ....’nn of the respective 

an entry. X, is made in the cell at th-mersect.on of the^ 

arrays. The rows and columns of t ^ diagonal band 

attempt to collect the cells with ^ every row 

from top left to bottom right accomplished with 

and in every column are adjacent. 1 _ individuals can 

an indecomposable matrix,'® the --j-j. ©f the rows and 

be described by a simple order in which the oroe 

that there exists an arrangc- 

An indecomposable matrix has the pfop , pair successiie 

mem of rows and columns such that the entries m ry 
roivs have entries in at least one common column 
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the order of the columns represent the ordinal positions of indi 
viduals and items on a latent attribute If such a pattern is not 
obtained, it may reflect error multidimensionahty, or unrepre 
sentative sampling 

An illustration of this form of analysis is given in Figure 8 corre 
spending to the data collected by Pick 2 illustrated in Figure 2 
The rows of Figure 8 each represent one individual from each class 
of phenotypic behavior 



Fio 8 Parallelogram analysis of Pick 2 data 


A significant aspect of this type of analysis is that it is com 
pletely subordinate to the data It does not map the data into a 
unidimensional continuum but in elTect, asks the data whether 
some of the conditions for a unidimensional continuum are satisfied 
As a consequence, the technique of Parallelogram Analysis does 
not necessarily yield a simple order It is a weak system of analysis 
in the sense discussed earlier 

THE UNFOLDING TECHNIQUE A change in the method of analy 
Sts occurs svhen we reach the Order 2 method of collecting data 
Here the cell entry, previously an X, is either a 1 or a 2, to represent 
an individuals first or second choice The analysis then consists of 
permuting rows and columns to form a parallelogram as in the 
analysis of Pick k data but m which the entries are digits as m 
Figure 9 The data analyzed tn Figure 9 correspond to tLse dins 
trated m Figure 6 

The results of such an analysts are identical to those of Pick 2 
except that with adequate sampling there are twice as many classes 
of individuals 
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Fig. 9. Untolding analysis of “Order 2’ data. 


The permuting of rows and columns it is deter- 

matrix is obtained which meets ^.p^ditions are met, then, 

;d that no such matrix exists. tijm reveals; (1) the 

» far as the power o£ that method o of a single underlying 

ivioT Tnnv Vip rprarded as a consequen ipact simply 


nai no suui luat-s*.. collecting data reveai^., 

the power o£ that method single underlying 

r may be regarded as a been at least simply 

item attribute on which the stimu i equivalence classes 

rdered; (2) the judges have -e behavior; (3) these 

orresponding to their equivalent p conditions are will 

lasses have been ordered. Exactly w a oaiygis of "Order 3 
le made explicit after a discussion o 

. , „i,.,xr;nT obtained from 


maae explicit alter a 

'r- ViphavioT obtained i»'-'*** 
Consider the classes of phenotypi -i^gle common latent 

>der 3" for five stimuli _ illustrated in Figure 7, 

tribute underlying the phenotypic^ ^ ristructed with cell entries 
r the analysis of such data, a matrix is co first, second, 

1. 2, and 3 in each row indicating an permuting rows and 

‘d third choice. The analysis then ^ i o_ 

'lumns to form a parallelogram, as m i^ ^^,ould possess, under 
The characteristics which such a ma underlying the 

conditions of a single common latent 
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choices of the individuals may now be made explicit for the general 

case of Order k " 

(1) The entries in each row (the integers from 1 to k) and in each 
column must be adjacent with no blanks 

(2) The entnes in the first row and the first column must monoton 
ically increase from left to right and from top to bottom re 
spectively 

(3) The entnes in the last row and the last column must monoton 
ically decrease from left to right and from top to bottom re 
spectively 

(4) The entries in all other columns must monotonically decrease 
and then increase from top to bottom 


A 6 C D E 
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Fic 10 Unfolding analysis of Order 3 data 

It IS now evident in what seme the Parallelogram Technique 
for the analysis of Pick A is a special case of the Unfolding Tech 
nique for Order k If *s were substituted for the integers in 
Figure 10 the data would correspond to Pick 3 instead of Order 
3 and discriminations among certain classes of individuals nould 
vanish In this instance the first four classes would collapse into 
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one class, the next two would collapse, -"d ‘he 'ast 

would collapse, leaving only three claves of rndtv.duals, correspond 

Th^:mparison reveals the difference in the searclungness 
of these methods of collecting data If an in ivi ua u , j 

had chosen il, C, and B as his first, second, and ^mes^the 

Unfolding Technique would reveal him as an abe 
in the sense that the latent attribute underlying his 
not the same as for the other individuals Under Pick 3,^ ^his 

individual would be indistinguishable ‘ ^ weaker condition 

tour classes that were collapsed together p p 1 a but the 

tor umdimensionality would be satisfied un er * e’ j jj 
stronger condition under ' Order 3 would not ^ 

behavior were obtained, the social scientist «nnld hen be fa 
with the dilemma of whether to regard this 
m his judgment or not Data collected at the power 
Order 3 do not contain such information f„„her imphca 

The analysis presented in Figure 10 ‘‘“J ^Xlvmg thrprefer 
hons with respect to the genotypic metric refations of the 

... !».■- 

for all methods of collecting data m . j [ j^ank Order 

Order k, • 2 ^ k ^ n 1 Ultimately for the Method of K 
«rrespondmg to Order n I,’ there is an integer m ) 
ihemairix . ^ »i,e data is 

Throughout all o£ these methods t le necessarily 

»«ainiamed No ordered metric or even sxmpi In tl,c 

obtained unless the data satisfy the ^^,11 usuall> not 

'domain of social psychological variables t le c social scientist 

taiufy these conditions and again the dilenima 
Jtises Either he Ins to choose between ns i . order holds 
bchcNcs or insists that an intcr\al scale or ** * pcrnnis n dis 
he has to use a method of collecting an m the 

^'action to be made between inconsistency "im 

l^^gmcnts of each indi\ idual , r discos cring lod 

The Unfolding Technique is a method 
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isolating a latent attribute underlying the preferences of a gioup 
of individuals or, in different terms is a method for disco\ering a 
genotypic scale The latent attribute, if present, is represented by 
an ordered metric with both stimuli and individuals on the Joint 
scale This is the only method available at present for analyzing 
choice behavior for latent attributes The method is now being 


extended to unfolding in multidimensional space (1) When this is 
done, It will permit the analysis of preferences into two or more 
latent attributes— an analogue of multiple factor analysis 

The handling of data collected by the Method of Paired Com 
parisons is more involved unless the paired comparisons are transi 
tive for each individual In the latter case, the data may, of course, 
be converted to rank orders and analyzed as above The theory for 
the analysis of intransitive paired comparisons Ins not yet been 
completed and there are several alternative psychological models 
all of which must be developed Similarly the treatment of data 
collected by the Method of Triads, using no stronger psychological 
postulate than that given on page 497, is somewhat more involved 
because at this level of data collection the data contain information 
on random error wuhm individuals These further developments 
cannot profitably be pursued here 

^"‘“’‘*'"8 Analysis of Quadrant I behavior collected by 
the Method of Successive Intervals or the Method of Equal Appear 
mg nteryals requires a simple modificalion m procedure In any 
row the djgit representing the ordinal rank of a pile will be asso- 
ciatcd with as many columns as there were stimuli placed in that 
in ividual The process of analysis is essentially similar 

fhvf or t e Order k methods of collecting data except 

must satnfv “f 'he conditions that the final matrix 

ZtrvZT.n I n “ ‘fe of the Method of Successive 

Inte™is in which the piles have no fiuifl metric relations 

the Meth ° rocthods of analyzing data collected by 

svstem ? ^ °‘“i. of a weak 

rnnldil ’"“l ■" ’'i "Sarded as paramount and 

a nnid.mens.onal analysts is obtained only if ,he data satisfy the 
necessary condttrons In general, as one goes down the scale of 
searchtngness to methods of collectrng data which are less searching 
II For a more detailed treatmem «e Coombs (8 Chap 7) 
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assumptions arc substituted com 

eral sense, are those necessary to make 

patible with the data collected rrtilprtmt? data 

It tsas mentioned earlier that the methods 
in Quadrant I constituted criteria for unidimensio y 
attribute It may now he seen more clear y m « la ^ , 

The techniques from "Pick n I" up to and - '■‘'■"S 3 

inaeasingly stringent criteria for a simp e or Order these 

up to and Lluding “Order n-l,” the ^ f'j^r but 

criteria successively demand not only the sa Thus, indi 

become more and more sensitise to st,muh but 

viduals may have the same simple oidc rmild not be 

different mltnc relations on them Such -"^.viduals could 
placed on a common continuum without vio j,scussed up 

The methods of analyzing the data of ' ^",r(cled 

to this point are methods for discosering hosvever, 

genotypic scales) underlying preferences ’„[ several 

may be used to construct phenotypic sc. y ' gcale is 

systems of analysis developed by Tbiirstone / p ti,em- 

a scale which best “represents ' the data but o 
shat IS, It does not in some sense go c rather attempts 

■nferences of a genotypic or explanatory nature but rathe 
to provide a simple desaiption of all the data for Quad 

Methods o£ analysis for arriving at , 1 ^ designed for 

1 data include the systems of analysis ^ Method of 

each ol the following methods of “ ^^e Intervals, the 

hqual Appearing Intervals, the Method ° Paired Compari 

Method o£ Rank Order (34), and the general of 

sons The Method of Paired Comparisons is Compara 

shese methods, and Thurstone has developed 

S've Judgment for the construction of scales rom for the 

These methods of analysis were designe y IV be 

^cahng of stimuli with respect to an attn u ' applied 

^^vior. and will be discussed below They may, rela 

to analyze Quadrant I behavior, appropriately 
‘onship of such an analysis to an analysis o ^jer the discus 

^'^Eold.ng Technique will be pointed out below unde 
of Group Scales 



520 The Analysis of Data 


Analysis of Quadrant IV Data 

thurstone’s scaling methods As previously indicated rela 
tive behavior involves a choice between stimuli In the preceding 
subsection the choice involved reference to an “ideal' stimulus and 
hence the behavior reflects the relative preferences of an individual 
In this section the behavior observed ivill be choice behavior but 


with reference to an attribute (Quadrant IV of Fig 1) Hence the 
behavior reflects the individual s judgments on the relative magni 
tudes of two or more stimuli in some respect 

The immediate objective of collecting such data is to study the 
stimuli Usually this involves using the data to determine the rela 
tive scale positions of the stimuli with respect to some attribute 
The most usual methods of collecting data to achieve this purpose 
are those of Thurstone the Method of Paired Comparisons, the 
Method of Rank Order, the Method of Successive Intervals and 
the Method of Equal Appearing Intervals 

The procedures followed to construct a scale from the data 


collected by any of these methods are well known and available in 
the literature (14) and will not be repeated here These are all 
strong systems which assume certain properties to hold for the 
information m the data, and the analysis yields an interval scale 
with the stimuli positioned on it The usual application of interest 
to the social scientist, Case V of the Law of Comparative Judgment 
applied to data collected by the Method of Paired Comparisons 
assumes that different individuals are replications of one another 
for the same stimulus situation Stimuli must be used which are 
relatively mdiscnminahle so that there is some disagreement be 
ween ju ges It is assumed that this disagreement is due to a given 
stimulus giving rise to a distnbution of sensation magnitudes As 
suming that th.s distribution is normal and that the L.ab.l.ty of 
he dtslnbutton of d.fferences between pairs of such distributions 
s ^nstant for all pairs (Case V) this latter variability ts then used 
as the source for a unit of measurement and an tnte^al scale may 


of ihT ,h n data collected by each 

of the other methods-the Method of Ranh Order, the Method of 
Successtve Intervals, and the Method of Equal Appearing Intervals- 
are essentially special cases of the Law of Comparative Judgment 
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and are listed m the order in rvh.ch a success.vely 
of assumptions are made or an increasing number of “ 

tmposed on the hehav.or by the method “^serving u ^These 
methods will always yield an interval scale un consists 

no error (i e , all individuals agree on every ]U gm 
entirely of .rror (te, individuals split 50 50 <>" ^;'-^VThmsXe 
It IS of interest to note in passing that the met o s 
require relatively indiscriminable stimuli w 
Technique is much more suitable tor completely discrimmable 
for relatively indiscriminable stimuli tn ron 

These Lthods of Thurstone are commonly 
struct an attitude scale with statements of opinion 
pro to ant. The scale so constructed has me 
but not the judges One then assumes that the sea 
tor an the /udgl or for a d.fferent group of „ 

teadmtmstered under a method appropriat ^ , operations 

to locate the individuals on the scale Two , ^nd 

tor collecting data are required to yield a Join 
‘he stimuli Lst be relatively indiscriminable m o d r that 
be some error variance to yield a unit of an inter 

No better methods have been devised or _ . ^ applied 
val scale for measuring attitudes The Unfo mg experi 

to such behavior may yield the joint distributio not 

taental operation hut it will be at best an or er j^nce has 

interval scale Furthermore, the present wn Yardstick 

shown that it is much more likely lo f«P'L ‘i'“l"“;eau.rL that a 
xists If, for reasons of belief or conve » traieht line by use 
social psychological attribute be measured on a s j^chniques of 
of the real numbers, it appears lo be assumptions and 

observation and analysis which embody sufficien P 

dassify sufficient data as error to ensure such a section, on 

tatr method of similarities In the pre presented 

otelhods of collecting data, the Method of Simi extended 

“ 0 method by which the Unfolding Tochmque contained 

to Quadrant IV behavior On the basis of the m udgments of 

0 such data, described in the preceding ..Qp judgments 

e individual may be converted into paire CO p regarded as 

tospecl to ak ideal m which f-.f.t pa.red comparison 

each stimulus m turn as his ideal 
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judgments are transitive for each ideal, then a rank order I scale 
IS obtained Such data may be analyzed as described for ‘ Order k 
m which / = n 1, each row of the matnv representing the rank 
order from a given ideal The entire matrix then represents the 
behavior of a single individual over the entire stimulus space This 
technique permits a relatuelj rigorous and intensive study of a 
single individual 

THE CROUP SCALE It has bccH pointed out several times that 
data collected by certain methods in Quadrant I (eg, the Method 
of Paired Comparisons) may be analyzed by the Unfolding Tech 
nique or by the Law of Comparative Judgment The relationship 
between these forms of analysis will now be shown 

If the Method of Paired Comparisons is used to collect data in 
Quadrant I, the judgments of individuals represent preferences on 
each pair of stimuli If the Law of Comparative Judgment is used to 
analyze such paired comparison data, the result is a scale which in 


some statistical sense (25) is most descriptive of the preferences of 
the individuals taken as a group The application of such a tech 
nique involves regarding preference as an attribute of stimuli (eg, 
preferability or popularity) and, in effect the experimenter is re 
garding such data as evaluation of stimuli with respect to an attri 
buie and placing it in Quadrant IV instead of Quadrant I 

Application of the Unfolding Technique to the same data may 
yield, if the 'ipproprnie conditions are satisfied, a Joint scale on 
which the stimuli and the individual judges are located instead of 
just the stimuli The unfolded J scale is a genotypic scale repre 
sentmg an inferred latent attribute underlying the preferences of 
the individuals The Law of Comparative Judgment solution is a 
phenotypic scale descriptive of the preferences 

In the more formal development of this general scaling theory, 
the individual s I scale, which is observed at best as a rank order 
may hypothetically be regarded as derived from a ratio scale of 
preferability for each individual Let us define a Group scale as 
a mean of all the I scales In the special case in which the I scales 
arise from a comrnon Joint scale, a Group scale is a Joint scale 
folded m the middle It can then be shown that a Law of Compara 
tive Judgment solution to preference data represents an approxima 
tion to such a Group scale These theoretical reHtions have been 
tested in seteral experiments and have been borne out Thus in 
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the case m uh.clt a single latent attribute underlies phenotypic 
preferences ol individuals, the Unfolding Technique yields a Joint 
Lie with individuals and stimuli located on it LaL approx 
parative Judgment solution represents the same 
imately in the middle, with only the stimuli 

This discussion of Group scales 1'=" ^ad expl.c t retej»« to 
preferential judgments collected by the Met lo ° 
sons and analyzed by the Law of Comparative J ^ classifying 
by the Unfolding Technique It slmuW ^ of Sue 

data collected by the Method of Rank Ord , intervals in 

cessive Intervals, and the Method of Equa PP*^” ic methods 
Quadrant IV and analyzing such data by I ®PP ^^cip scale Data 
simply represent different “PP''°’‘’"’““°"" ^jsq be^analyzed by 
collected by any of these latter methods y ^ 

the corresponding case of the Unfolding Tec conditions 

mensional Joint scale obtained under the approp collected 

The theory and the computat.onal oLer. 

by the Method of Paired Comparisons the et o ^ Appear 
die Method of Successive Intervals, or the i e jy pjcthod 

‘ng Intervals and analyzed by their appropria e 14 ^ 

of analysis make no distinction between tas ® however, 

methods, all behavior is task B Only certain 0 ^vhen 

“ay be classified in Quadrant I and un ° ^ evaluation 

iliese methods are used to observe task /I 
of stimuli with respect to an ideal 


Analysis of Ouadianf II and III Data 

data associated 

As was pointed out in the preceding sec ‘ ’ , Method of 

''dh these two quadrants are those co ecte experi 

^mgle Stimuli It was further pointed out t at ^gnis b> 

dentally independent successive responses to a ” jtem) 

individual choosing one alternative as ns , choice 

constituted a special case, " Pick 1.' of the Method 

Prom the point of view of the inform items, asso 

two types of items were distmguishe ^ , terns, associ 

‘^ated With Quadrant Ila and HI. and nonmonotone ^ 

’“d with Quadrant Ilb The information co ,he ideal 

"ms, in the particular case of two alternatives. 
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of the individual is greater or less than the position of the item on 
the genotypic scale 

In the case of the nonmonotone items, the information con 
tamed in the data is which of the alternatives to an item is nearer 
the individual s ideal in any direction, rather than on one side of 
him as in the case of monotone items Consequently, if the en 
dorse alternative is ‘ too far away in any direction, the phenotypic 
behavior is “not endorse, and hence there may be several distinct 
kinds of genotypic individuals responding identically phenotypically 

The kinds of data usually classified m Quadrants Ila or III 
include mental lest data, attitude or questionnaire items w ith Likert 
type alternatives, items of a cumulative nature, and rating scale 
data The kinds of data usually classified in Quadrant 11b include 
statements of opinion ranging from pro to anti, administered by 
the Method of Single Stimuli with the alternatives “agree ’ or “n<’ t 
agree 

Lazarsfeld is in the process of constructing two related ssstems 
for the analysis of Method of Single Stimuli data These systems are 
his Latent Distance Model and Latent Structure Analysis These 
systems are complex and as yet relatively undeveloped so m many 
instances they are not practicable Although these systems are not 
computationally feasible this constitutes only a transient difficulty 
Lazarsfeld s system is actually a theory of theories a raetatheory, 
of methods of analyzing irrelative behavior Lazarsfeld s system is of 
such generality that it provides a theoretical framework within 
which specific methods of analyzing data collected by the Method 
of Single Stimuli may be understood as special cases Both Guttman s 
scaling theory and mental test theory will be presented in this 

rnntfvf * 


Viewing various methods of analysis as special cases of a more 
general theory reveals the fact that an experimenter in selecting a 
method of analysts is selecting a theory about behavior The data 
are either asked to satisfy or {creed to satisfy the theory, depending 
on the strength or the postulat.onal basts of the specific theory 


i'’See for eximple Stouffer (31 p Hi) 

13 This distinction is an arbitnrv one Insed on the use of discontinuous 
trace lines in the Latent Distance Model and continuous trace lines in Lntenl 
Structure Analysis The disimct.on is convenient here because Guttman s staling 
theory ,s a special case ot Ihe fim ,„d mental test theory a special case ot Ihe 
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underlying the method of analysts selected The "" 

objective of these techniques is to convert the in ,^n,ques 

data to positions on an ordinal or interval scale 

of analysis differ only m the properties or constrai P 

on the information m the data, the techniques -nge torn very 

weak systems, which make the least assumptions a 

yield a simply ordered scale, to very strong systems, which ma 

sufficient assumptions to guarantee an interva sra ^ 

LAZARSFELD-S LATENT DISTANCE MODEL T>US ™Odel P^ 
cally designed for the analysis of Q“ 2 d'-ant continuum is 

data In the simplest form o£ this model, 

assumed to be dichotomized at some point y , . , * of en 

individuals on one side of that point have a P™ jc 

dotsmg or passing the item , and all individuals on the 


l.On 

Probability of 
'lie response pass, 
°9ree, endorse, .5^ 
yes, etc 


onH (b) Lazarsiclu 

kic 11 Example of (a) Cuttinan type non' 
type Item 

that point have the probability i to more than 

“cm {cf Fig 11) This model is readily 

*"0 latent classes for other than dichotomous i . technique 

^t IS immediately apparent that Gutim^ s model re 

“ ^ tpeaal case of the dichotomous item ' 

that be zero or one (cf nnahS’S requires per 

In different terms, Guttmans s)stem ° , whereas Larars 

ta y reliable items (perfect internal consis p. pcrmntmp; 

'•'-l «n handle less than perfectly reliable items 
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of the indiMdual is greater or less than the position of the item o 
the genotypic scale 

In the case of the nonmonotone items, the information cor 
tamed in the data is which of the alternatives to an item is neare* 
the individual s ideal in any direction, rather than on one side o 
him, as in the case of monotone items Consequently, if the ‘ en 
dorse alternative is "too far away ’ m any direction, the phenotypic 
behavior is "not endorse,’ and hence there may be several distinct 
kinds of genotypic individuals responding identically phenotypically 

The kinds of data usually classified in Quadrants Ila or III 
include mental test data, attitude or questionnaire items with Likert ' 
type alternatives, items of a ‘cumulative’ nature, and rating scale ' 
data The kinds of data usually classified in Quadrant II& include 
statements of opinion ranging from pro to anti, administered by 
the Method of Single Stimuli with the alternatives "agree” or ‘ not 
agree" 

Lazarsfeld is in the process of constructing two related sistems 
for the analysis of Method of Single Stimuli data These systems are 
his Latent Distance Model and Latent Structure Analysis These 
systems are complex and as yet relatively undeveloped so in many 
instances they are not practicable Although these systems are not 
computationally feasible, this constitutes only a transient difficulty 
Lazarsfeld s system is actually a theory of theories, a metatheory, 
of methods of analyzing irrelative behavior Lazarsfeld’s system is of 
such generality that it provides a theoretical framework within 
which specific methods of analyzing data collected by the Method 
of Single Stimuli may be understood as special cases Both Guttman s 
scaling theory and mental test theory will be presented in this 
context 


Viewing various methods of analysis as special cases of a more 
general theory reveals the fact that an experimenter m selecting a 
method of analysis is selecting a theory about behavior The data 
are cither asled to satisfy or forced to satisfy the tlieory, depending 
on the strength or the postulat.onal basis of the specific theory 


12 See for cximplc SloiilTcr (31 p Uiy 

nTh„ an a.b.tr.,, toed on .ho .„o of dncon.mno,,. 

.race ], no, .n ,1,0 U on. Ilmanoo Modol and „„„n„„„, ,„ce hne, ,n I.-dcn, 
S.n,r,n,o \nal,-,n Tho dm.nc.on con.on.om horo hocan.o Cndman, ,cal,ng 
.hoor, „ a ,poc.al ca,o of ,l,o r„.i md „onIal .hoory a , pec.nl case of iho 
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error or mconsistenqr in behavior, Lazarsfeld's system can yield a 
solution when Guttman's cannot Lazarsfeld’s solution provides a 
set of at least two latent classes on the genotypic level and a proba 
bility that each response pattern is associated with each latent class 
When the data satisfy the conditions for a simply ordered scale, 
Lazarsfeld’s system reduces to Guttman’s in that the probability of 
a specific response pattern s being associated with a specific seg 
ment of the hypothetical continuum is zero or one 

The Guttman scalogram technique is used to analyze Quadrant 
lla data to determine whether the conditions for a simply ordered 
scale are satisfied The method is closely related to the Parallelogram 
Technique m that a matrix is constructed in exactly the same man 
ner and the rows and the columns are permuted But inasmuch as 
these Items are monotone items instead of nonmonotone, the final 
matrix, if ordinahiy is satisfied, has all the X’s on one side of the 
diagonal (including the diagonal) and all the blank cells on the 
other side of the diagonal For this reason this method of analysis 
may be called Triangular Analysis 

Guttman's scaling theory constitutes a strict adherence to the 
logical structure of an ordinal scale If all the data do not satisfy the 
conditions, he calls the result either a quasi scale or nonscale type 
The latter is really a partial order If an ordinal scale is insisted 
upon, part of the data must be rejected-either individuals, stimuli, 
or both Otherwise the latent distance model, which can classify some 
of the data as error and still yield an ordinal scale, must be used 
The result of a Guttman analysis on data which satisfy the 
necessary conditions is an ordinal scale with the stimuli and the 


response patterns of the individuals simply ordered The technique 
as conventionally used is applied to dichotomous items If the items 
coritam more alternatives than two. the surest way to find an ordinal 
scale IS to group the alternatives to form that dichotomy ishich best 
happens to satisfy the conditions for an ordinal scale This procedure 
takes advantage of every error, random or biased, m the dam but has 
common y been followed because of the generally unsatisfactory 
results of trying to scale all the alternatises 

It IS possible to show that the scaling of the alternatives for 
different Items by Guttman's technique requires an additional con 
dition above the ordinal propeay of the scale It is not feasible 
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to develop this here, but the general principle is that it is not 
the alternatives which should be scaled but the section line between 
two adjacent alternatives of an item The alternatives may them 
selves appear to be scaled only if the alternatives of each item mesh, 
like the teeth in gears, with the alternatives of the other items This 
condition can be called "orderly interlocking* If this condition 
does not hold, the conditions for an ordinal scale may exist but no 
ordinal scale will be found m the data unless the boundaries of the 
alternatives are scaled instead of the alternatives themselves Thus, 
an Item with five alternatives has four scale positions, one between 
alternative a and b, one between b and c, c and d, and d and e In 
this manner it is also possible to handle, in one scale analysis ques 
tionnaire items which have varying numbers of alternatives 

It has previously been pointed out that every item in the Method 
of Single Stimuli, task A, monotone stimuli is really a disguised 
mating scale The experimenter must select an a piiori order among 
the alternatives to the item The individual in responding, simply 
informs the experimenter of his location on the rating scale Methods 
of analyzing Quadrant Ila data are consequently simply methods 
of analyzing the task A responses of a number of individuals to a 
number of rating scales Hence, from the abstract point of view of 
nieasurement theory, rating scale data can be mapped into Quadrant 
Ilfl and hence the methods of Gunman and Lazarsfeld are applicable 
to the analysis of rating scale data 

ThfTfxs £iOi*rjbararteristic of Gunmans scalir\g techm9Ufi wJurh 
IS either an advantage or a disadvantage depending on one s point 
of view If the data do not meet the necessary conditions for an 
ordinal scale, one will not be obtained This is a good characteristic 
if one IS interested in the study of behavior of individuals and the 
nature of attributes, for the technique does not force a more power 
ful S)stem on the data than the data satisfy On the other hand, if an 
ordinal scale is demanded, this characteristic is a disadvantage In 
the latter case, if an ordinal scale is not found, the alternatives are 
to reject individuals or stimuli or change the responses of some 
mdividinls to some items— i c , classify them as error 

Gunman, in his general theory, makes much of a theory of com 
ponenis whicli are successive sources of the variance of individuals’ 
behavior He particularly makes much of the psjcliological inierprc 
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error or mcorrs.stency m behav.or, Lazarsfeld s system can yield a 
solution when Guttman's cannot Lazarsfeld s solution J 

set of at least two latent classes on the genotypic level and a proba 
bility that each response pattern is associated with each latent class 
When the data satisfy the conditions for a simply ordered scale, 
Lazarsfeld s system reduces to Guttman's m that the probabdity ol 
a specific response patterns being associated with a specific seg 
ment of the hypothetical continuum is zero or one 

The Guttman scalogram technique is used to analyze Quadrant 
lla data to determine whether the conditions for a simply ordered 
scale are satisfied The method is closely related to the Parallelogram 
Technique in that a matrix is constructed in exactly the same man 
ner and the rows and the columns are permuted But inasmuch as 
these Items are monotone items instead of nonmonotone, the final 
matrix, if ordinality is satisfied, has all the X’s on one side of the 
diagonal (including the diagonal) and all the blank cells on the 
other side of the diagonal For this reason this method of analysis 
may be called Triangular Analysis 

Guttman's scaling theory constitutes a strict adherence to the 
logical structure of an ordinal scale If all the data do not satisfy the 
conditions, he calls the result either a quasi scale or nonscale type 
The latter is really a partial order If an ordinal scale is insisted 
upon, part of the data must be rejected— either individuals, stimuli, 
or both Otherwise the latent distance model, which can classif) some 
of the data as error and still yield an ordinal scale, must be used 
The result of a Guttman analysis on data which satisfy the 
necessary conditions is an ordinal scale with the stimuli and the 


response patterns of the individuals simply ordered The technique 
as conventionally used is applied to dichotomous items If the items 
contain more alternatives than two, the surest way to find an ordinal 
scale IS to group the alternatives to form that dichotomy which best 
happens to satisfy the conditions for an ordinal scale This procedure 
takes advantage of every error, random or biased, in the data but has 
commonly been followed because ol the generally unsatisfactory 
results of trying to scale all the alternatives 

It IS possible to show that the scaling of the alternatises lor 
1 erent items by Guttman's technique requires an additional con 
tlition above the ordinal property of the scale It is not feasible 
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tations which he gives to the first and second components The first 
component is the simply ordered scale o£ stimuli and individuals 
The second component is a U shaped function which he interprets 
as ‘ intensity ’ He suggests that individuals at the extremes of the 
attitude continuum feel more strongly about their attitude than 
those in the middle region, in the neighborhood of indifference This 
was an early suggestion of Katz (18) and is a reasonable psychological 
hypothesis But the interpretation of the second component as a 
measure of this intensity of feeling must be very carefully validated 
experimentally As indicated by the class of data to which Guttman s 
theory applies, mathematical analysis into components is as valid 
for mental test behavior as for attitude data collected on monotone 
Items These same mathematical components would be found in the 
behavior of individuals on an arithmetic test, for example, but the 
same psychological interpretation of intensity of feeling about one’s 
arithmetic ability would not necessarily follow The interpretation 
to be given components may be different for different domains of 
behavior 


lazarsfeld’s latent structure analysis Instead of regarding 
the underlying attribute as having discrete steps or classes, in Lazars 
felds Latent Structure Analysis (20), the underlying theory is that 
of a continuous gradation in the latent attributes The analysis, as in 
the latent distance model, yields discrete latent classes, but the basic 
psychological theory requires an underlying continuum Latent struc 
ture analysis is Lazarsfeld s general system for the analysis of non 
monotone items. Quadrant II6 data, but m which monotone items 
are a special case It is in this sense that Lazarsfeld s general theory 
is a theory of theories for irrelative, task A, behavior In principle 
is t eory is applicable not only to monotone and nonmonotone 
terns but also to combinations of them in the same test or ques 
tionnaire ^ 

Analysis has the basic concept of a trace line 
rdTiinair,' '' ‘'™ the probability of an 

favnrahl P°*tt °ti the latent attitude continuuni responding 

favorably to the given item (Fig 12) ^ 

linn f ttages of the development of the theory, the selec 
ion of a function (straight hues, parabola etc) for the trace hue 
was an a prion decision, but there is an expectation that this will 
become objective and unique (,2) This system is applicable to 
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a score of two could be obtained by passing any t^vo items PPtP, 
PrPP, PPPr, PPPP, etc 

If the conditions for a Gunman scale hold, a one to one cor 
respondence exists between a response pattern and an integer 
corresponding to an ordinal position on a unidimensional con 
tmuum To convert such a scale into an inter\al scale, an assumption 
must be made such as that the increments m ability between the 
ordinal positions of adjacent items on the scale are equal 

When conditions for a Guttman scale are not met, there is more 
than one response pattern mapped onto the same integer and the 
scale IS represented by a partial order, what Guttman calls a non 
scale type The rationale for mapping such a partial order on an 
interval scale is somewhat more involved, since it is based on a 
concept of error and random fluctuations 

The concept of error commonly used m test theory corresponds 
to the assumption that the trace line of a mental test item is the 
integral of a normal curve (cf trace line c in Figure 12), although 
Carroll (5) is currently working on the theory of a linear trace line 
for mental tests If the items were perfectly reliable, the variance 
of the normal curve would be zero and the trace line would take 
the form of that shown for item a in Figure 11 The rationale 
underlying test theory is critically evaluated by Reese (26) and 
Thomas (32) 

Likert (21) has suggested a technique for arriving at a score on 
an attitude questionnaire which is a simple extension of mental test 
mg technique Likert’s system is necessarily confined to monotone 
Items but is an extension of the scoring methods of mental testing 
in that degrees of “endorsement” are obtained in the data The 
technique is to take a statement of opinion sufficiently extreme so 
that It cannot act nonmonotonically— that is, so that there are not 
likely to be any people so extreme that they would reject the item 
for not being sufficiently extreme To such an item, a degree of 
endorsement is obtained, typically from strongly agree to strongly 
disagree The simplified scoring method is to map these alternatives 
into the integers-eg, l to 5-so that 1 represents the extreme pro 
answer and 5 the extreme anti The individual s score, then, may 
be simply the sum of those integers into which his responses have 
been mapped Obviously this corresponds identically to scoring a 
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underlying test theory is critically evaluated by Reese (26) and 
Thomas (32) 

Likert (21) has suggested a technique for arriving at a score on 
an attitude questionnaire which is a simple extension of mental test 
ing technique Likerts system is necessarily confined to monotone 
Items but is an extension of the scoring methods of mental testing 
in that degrees of ' endorsement’ are obtained m the data The 
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that It cannot act nonmonotomcally— that is so that there are not 
likely to be any people so extreme that they would reject the item 
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general thcoiy The advantage of an abstract approach to methods 
of collecting and methods of analyzing data, from the point of view 
of measurement theory, lies m its permitting the comparison of dd 
Cerent techniques and their interrelations, it also permits general 
izing their applicability 

For example, from this abstract point of view, the data fiom cer 
tain learning experiments can be classified as Quadrant I In data 
A conditioning experiment in the abstract corresponds to a mental 
test given backwards The conditioned stimulus and the unconcU 
tioned stimulus together may be regarded as the stimulus situation 
If the response occurs after the conditioned stimulus and before 
the unconditioned stimulus, the item is ‘ passed ’, if not, it is ‘ failed 
The stimuli are presented in the order of the most “difficult item 
first and, as conditioning progresses, the items become ‘ easier ‘ The 
analogy is now clear If one wanu to use the strong system of test 
theory, each individual s performance is mapped into the integers 
by counting trials to learn This requires the assumption that the 
increment in learning between all pairs of successive trials is con 
stani within and among individuals 

In conditioning experiments, the presentation of stimuli may be 
continued until the individual gets a certain number of them right 
in succession In mental testing the corresponding concept would 
be to require the individual to get a certain number of items xurang 
in succession In general, any special technique designed for the 
analysis of learning data, such as a sequential type of analysis, would 
then be applicable in principle to the analysis of questionnaire 
and mental test data which also meet the abstract conditions of 
Method of Single Stimuli task A, monotone stimuli On the other 
hand, any technique designed for the analysis of such Quadrant Ila 
data IS in principle applicable to the analysis of data from such 
learning experiments Hence, Lazarsfeld s latent distance model 
or latent structure analysis could be used to study learning 

Lazarsfeld s system gams much of us importance from the fact 
that It is designed for the analysis of data collected by the Method 
of Single Stimuli, and this method is probably more widely used 
particularly m social science, than any other The method is widely 
used in public opinion surveys certain sociometnc data also fall 
into this class of behavior When, for example, the members of a 
group are asked to indicate which individuals in the group they 
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CHAPTER TWELVE 


Distribution-free Statistical 
Methods and the Concept of 
Power Efficiency 


Keith Smith 


Social scientists today are more aware perhaps than any other 
scientists of the restrictive nature of a priori assumptions concerning 
data But although they sometimes follow tortuous routes to avoid 
“common sense” assumptions which go beyond the theoretical frame- 
work in which their studies are embedded, nevertheless, new measures 
and measuring devices are constructed and applied to two groups, and 
tests of the hypothetical difference between the groups are used which 
are based enltrely on the assumption that the measured values are 
distributed according to the normal curve over both populations in- 
volved Thus, although they are very sensitive to assumptions about 
what might be called their “real world,” social scientists are prOne to 
be insensitive to assumptions in the statistical systems in which they 
embed their data The measurement or statistical systems into which 
data are mapped constitute an integral part of the theory and assump- 
tions about the “real world ” 

For example, the crux of a sophisticated theoretical question about 
the “real world” may he on a base which requires that, had the 
instrument been applied to all of some universe, usually hypothetical, 
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able to evaluate or compare procedures in order to be able to select one 
that in some sense best satisfies the experimenter’s objectives 

The comparison of test iJotisticj is based on the question “How 
often will It give the right answer^” For example, if a difference exists 
between the means of two populations, will one test more often yield a 
significant difference than another test for random samples of the 
same size‘s The comparison of estimation statistics, succinctly referred to 
as estimators, is based on two questions How often will it give the nght 
answer, and how much information does it give about the population 
parameter’ 

The “How often” question is answered in the same way for both 
classes of statistical procedures by means of what is called the f-ower 
curie This wiU be discussed m general terms in the second section of the 
chapter In the third section, where various nonparametric tests of 
significance are discussed, the concept of the power curve will be used 
for the purpose of comparing the nonparametric tests with conven- 
tional parametric tests 

The second question used to evaluate estimators, “How much 
information does it give about the population parameter’” pertains to 
the size of the confidence interval which brackets the parameter Other 
things being equal the shorter the confidence interval an estimator 
yields on the average, the better the estimator The fourth section of the 
chapter will deal with estimators, unfortunately, however, very little 
has yet been accomplished on the problem of comparing nonparametric 
estimators 


CRITERIA FOR THE COMPARISON 
OF STATISTICAL TESTS 

As a consequence of the fact that alternative statistical procedures 
ha\c been developed which can be applied to the same data, it becomes 
both necessary and desirable to provide a rational basis for choosing 
one procedure instead of another In this section, a number of criteria 
for making this decision will be presented, with particular emphasis 
on the concepts of the power and the power efficiency of a statistical 
test This choice of emphasis is due to the general unfamiharity of these 
concepts to social scientists and to the significant role they should play 
in the choice of a statistical lest 

Before the criteria arc presented and discussed, it will be desirable 



DistribuHon>free Methods 


539 


to review briefly certain charactcnstics of statistical tests in general. 
In this pre‘;entation, a knowledge of clctneniary sampling theorv will 
be assumed. 


Dejinilion of Slalistical Tests 

A statistical test is a formal mechanism, based on probability, for 
arriving at a decision about the reasonableness of an assertion The 
assertion is called a hypothesis, and any value (usually a number) 
obtained from a sample of data is a test statistic The mechanism makes 
use of the one or more obtained values (test statistics) to arnve at a 
probability statement about the assertion But the mechanism almost 
always makes use of more than that, and these additional things are 
other assertions about the population from which the sample was 
drawn {eg, normally distributed), the manner in which the sample 
ivas dra\\n (c g , randomly), etc 

From the point of view of the person making use of the statistical 
lest, the assertions miolved are of two kinds One is an assertion 
directly related to the purpose of the investigation this is an assertion 
'^hlch vs to be tested and is called a hypothesis All the other assertions 
are those which it is necessary to assume to make a probability state- 
nient This second set of assertions is called the model All probability 
statements about a hypothesis are preceded, implicitly or explicitly, by 
the qualifier “If the model used was correct, then 

It should be clear that the weaker the assertions that define the 
model, either by Nirtue of fewer assumptions or less restrictive assump- 
tions, the more ceneral the conclusions On the other hand, the 
the moa’ef— i e , the more assumptions' buii’C into if— tfie more 
powerful will be the test of the hypothesis 

As an illustration, a statistical hypothesis might be “Population A 
has a mean equal to the mean of population B ” Such a hypothesis 
Contains no normality statements and, in fact, no statements about any 
characteristics of the populations other than the means If a test of this 
hypothesis is used which contains an assumption of normality in the 
parent populations, normality becomes a part of the model and 
normality is not tested 

Two charactcnstics of a statistical test ha\e been discussed (1) the 
niodel, the assertions which are assumed to be right, and (2) the 
statistical hypothesis, the assertion which is to be tested, called a null 
hypothesis A third characteristic always required m a statistical test is 
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the hs-po hesii or of h%'pothe5«s which arc altemati\e to the null 
h\poiher^ These altername hypotheses are clj.ais part of the test, 
whether explat O’- impliat. These coasumte the assertion that is 
acccp ed if the null hvpothesis is rejected 

From ih*s pomt of \ac'\, it u evident that there are two types of 
erro*- which are possible m arriving at a decision about the null 

h%-pothesis Tyfgl accepting the altemamc hvpothesis when the null 

h\-pothesis IS true Ty* II accepting the nUl hvpothesis when it is 
false 

There is an in\cr^ relationship bctisecn the hlciihoods of mak i n g 
these tivo tvp>es of emr If the null hypothesis were alw-ai*s accepted 
rctrardless of the sample test statistic, the probabilit\ of a Ti-pe 1 error 
would be zero, but there would be a maxunum likelihood of a T\'pe II 
error Correspandini^h , if the buU hx-potbais \tcrc alifars rgccted, 
the dant’C’- of a Ts-pe II erro'- would vanish, but there would be a 
maximum likelihood of a Tvpe I error \ compromise must be reached 
between these t>>o dangers, and vanous nauscical tests offer the pos 
sibihtN of different comp-omises. This is the problem for which the 
concepts of the p xgr o' a stamacal test and its p-'^er are 

rel ex-art 


Pejrr erd Pc^'t 

The concepts of pow er and pox\cr efficiencx will be mtroduced bx 
XX av of an illustration 

A COitP ARISON OF TV>0 STAilSiiCAl. TEXTS A fJi mm nn problem 

IS a test of a hx-poihcsis about the value of the rnpar> of a population. 
The null hx-poihe$is might be that the mean of the population from 
xxhich the sample x\-as draixm is 4 An altematixe h\-polhcsis mi»ht be 
that It IS a, and an infinite class aliemaUNc hypotheses all used at 
once micht be that it is creater than 4 

The simplest posnb’e case of a staosiical test is whether a popula- 
tion has one or the other of txvo specific x-alucs, all the other char- 
actcmucs of the population hanng been specified in the model Let 
th- assertions that the population is nom^al wath x-anarcc equal to 
on* cons’atute the model Let the null hx-pothesis be 
H the mean is l 
I^t the ihemauxe hypothesis be 

Hi the ir*an 15 2 
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. j tVsA hnijis of the model and the 

A sample of size 1 is dra^\n an , „ ,o to be made between the 

: X^e — n specRed by .he 

model IS illustrated in Figure 1 . 



=3"“ ‘“f . „e shall choose a number A larger 

To use the one-tailed , , s jf ^ur sample value is 

than 1 (the mean under the nu yp hypothesis and, of 

larger than oth:!, I! that the mean 

- -- to accept the “"‘F ^ ,hall agree to accept the 


course, to accept the only ghgll agree to accept the 

1 , 2 If the sample value is , way as to make the 

null hypothesis A must be c ose 

probability of a Type I error ^ ^ a table of normal 

under the curve Ho to the . c xhe probability of making a 

Type If error is the area under . „„e 

I « , the proportion of times w probability is 0 74 

Agam, from a. ahlecf norma, area^ , 3nd C, 

To use the two-tailed ^ hv po.hesis If our sample 

equally distant from the mean u , aarce to reject the null 

% akic IS farther from 1 than these t"* ,s nearer 1 than 

hypothesis and accrpl the only ® . ^,5 yhe area under Ho to the 

B and C are, we accept the left of C must also be 

right of B must be 0 025, and the B = 1 + 1 96 = 2 96 

0 025 to make the tot^ O^a And p„babil..v of a Type II error is, 
<ind C — 1 1 
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of course, the area between -0 96 and 2 96 under the curve H, This 

area is 0 83 e 'r t 

Thus, although the two tests have the same amount of Type I 
error, the two tailed test of this hypothesis has more probability of a 
Type II error, and in this sense is the poorer of the two tests against this 
alternative In this simple example, it should be apparent that values 
to the left of C are poor evidence for Hi The two-tailed test is used 
here only to illustrate the fact that the rejection level of a test is not a 
sufficient basis for its evaluation 

POWER We are now ready to introduce the concept of power 
The power of a statistical test against a specific set of alternative 
hypotheses at a specific significance level is given by the equation 

Power == 1 — (Probability of a Type II error) 

Power might alternately (but equivalently) be defined as the proba- 
bility of rejecting the null hypothesis when the alternative hypothesis 
IS true In Figure 1, the power of the one tailed test is the area to the 
right of A under the curve H\ In tests of means from normal popula 
tions the concept of power is synonymous with the Fisher concept of 
“amount of information” (3) 

Research workers sometimes use certain tests because the tests 
are “conservative,” meaning that they are less powerful than others 
that might be used The discussion above exposes this as a rather 
peculiar sort of conservatism Testable theoretical deductions are rare 
enough without loading the dice against them, but this is what the 
research worker does if he wishes to reject the null hypothesis Further- 
more, the “conservative” test is in no sense conservative if the research 
worker wishes not to reject the null hypothesis (as m homogeneity of 
variance tests m analysis of vanance) 

In our example above, the power of the one-tailed test is 0 26, 
whereas the power of the two tailed test is 0 17 Thus, the one-tailed 
lest is “more powerful ’ than the two tailed test at the 5-percent level 
against the hypothesis that the mean of a normal distnbution with 
vanance 1 is 2, when the mean under the null hypothesis is 1 So far, 
all these qualifying phrases are necessary and illustrate the caution 
essential when statistical tests are compared It can be shown, in this 
simplest of cases, that, at any significance level, whenever all alternative 
hypotheses state means on one side of the null hypothesis mean, the 
one-tailed test will be more powerful than the two-tailed test 
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To show that the power of a test rs 
hypotheses, let us take as another -ample the same stt^ 
that here the alternative hypothesis “ ® determined 

this case the values of A, B, and re ^ 2 ^^ 

by the nun ‘'VP°*7/Xwe“n ^and C is 0 52 Consequently, the 
"t^Tne-tld te;t is 1_- 0 36 = 0 64. whereas the power of 
the two-tailed test is 1 0 52 0 4 



f! 2 The dislnbulionsspeciHcd by the alternatives//, and H! 

One can plot, for this model “^mficanceje«^^^^^ 
each test against a ‘‘yP°*“‘®7!»po„cr curves” for the one-tailed and 

The curves Obtained Will be the power c 

two-tailed tests (Fig 3) 



Tio 3 Pow cr cur\ cs of tnc on 

1 Kr* Init one alternate c. 
u IS, of course, not ''-rhate' ihe' a.lematnes 

Another test using the same 
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Hi mean is 2, Hi mean is 3 Here, also, the one tailed test is more 
powerful against both alternatives, since it is more powerful against 
each singly The difference in tests would only be that rejecting Ho 
would not imply accepting a single alternative but would imply only 
accepting the pair of alternatives 

However, if the alternatives had been Hi mean is 0, and Hs mean 
IS 2, a considerable difference obtains As Figure 3 indicates, the one- 
tailed test has almost no power (0 004) against the hypothesis that the 
mean is zero, whereas the two tailed test has a power of 0 17, at least, 
against both alternatives 

A still more complex test would be one with the infinite number 
of alternatives Hi mean is greater than 1 Again, the two-tailed test is 
less powerful than the one-tailed test, since it is less powerful for all 
alternatives 

Last, but most often used, is the test with the infinite class of 
alternative hypotheses the mean is not 1 As would be expected from 
Figure 3, the two tailed test would be used, since it is only slightly less 
powerful on the right of the null hypothesis, but extremely more 
powerful on the left 

POWER AND SAMPLE SIZE The preceding illustration was entirely 
in terms of a sample of size one The illustration can be generalized by 
using, instead of the sample value, the number (XViO/V where a is 
specified in the modtl as the standard error of the normal distribution 
involved, n is the sample size, and X is the sample mean As n increases 
the test statistic, {X's/n)/^ increases and, by Figure 3, the power of 



Fic 4 Power curves of the two tailed test with varying sample size 
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the test increases also Figure 4 illustrates this mcrea<e m power of the 
two tailed test of the mean for samples of size 1, 4, and 9 from normal 
populations with unit variance, and fio is the mean under the null 
hypothesis 

For almost every statistical test in use, including all those to be 
discussed in this chapter, increasing sample size increases the power of a 
statistical test 

To summarize, (1) the power of a statistical test is the probability 
of rejecting the null hypothesis when it is false, (2) power is relative 
to the model employed and to the alternative hypotheses (possibilities) 
entertained, (3) as a general rule, the power of a statistical test increases 
with sample size 

POWER EFFICIENCY Unfortunately, there are considerations other 
than those of power that must be made in the choice of a statistical test 
iS the test simple computationally^ Is the model (set of assumptions) 
required for this test ‘‘true to hfc”^ Leaving the first question for the 
moment, let us consider how the second might be answered 

It was stated earlier that the weaker the assertions constituting 
the model, the more general the conclusions but the less poiverfuJ the 
statistical test This is very generally true /or a gwen sample size but not 
necessarily so if the tests use different sized samples Test A may be 
better (more powerful) than test B for samples of size twenty, but test 
B may very well be more powerful with a sample of size uventy than 
test A IS with a sample of size ten In other words it is necessary to pay 
for increased generality of conclusions with a larger sample Power 
efficiency is a measure of how much one has to pay m any specific case 
If test A with a sample size Nji has the same power as lest B with 
the sample size Ng, where test B is the most powerful test (know n or 
bypothetical) for No observations, then test A has power efficiency 
^^0{Nb/NX) percent If test A requires a sample of 10 to be as powerful 
^ lest B with a sample of size 6, then test A has power efficiency of 60 
percent The question becomes “Is it worth the extra expense of taking 
a sample this much larger to arrive at more general conclusions'* 

The preceding paragraph ts adequate when there is only one 
ahcrnaiive hypothesis ^Vhcn (here arc more than one, the phrase “is 
powerful as’ loses meaning Test A may be more powerful against 
ahcrnaiuc one but less powerful against alternative two How, in (his 
tnsiancc, can vse say one lest is more powerful than the other, or as 
powerful as the other’ In Figure 5, winch test is more powerful’ 
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No unique or “best” answer exists Many rational answers can 
be found, two of which can be discussed here A coramonly used 
answer is the equaharea criterion In Figure 5, test A, by this criterion, 
would be as powerful as test B if the ruled area were equal to the cross- 
hatched area Another means of equaung power curves is to adjust 
sample sizes so that the power curves intersect at some central power 
level (say 0 50) The two definitions do not yield essentially different 
ansis ers ordinarily, and the two tests m Figure 5 are a case in point 



Fio 5 A portion of the power curves of two h>pothetical tests 

A second and last important complication is that power efficiency 
Itself may vary for different sized samples Test A may have a power 
efficiency of 0 70 with samples of 5 and a power efficiency of 0 95 with 
samples of 100 A test s power may increase rapidly with sample size 
while the power of the most powerful test is increasing only slowly 
To summarize, then, power efficiency is a measure of the power of a 
staiisticil test relative to the most powerful test possible Although 
power efficiency may be defined in several ways, the definitions are 
roughly equivalent Finally, power efficiency is not completely inde- 
e?f size in aB cases 


TESTS OF HYPOTHESES WITH 
ORDER STATISTICS 

The rest of this chapter will be devoted to a discussion of a nu mber 
ol nonparametne tests and estimation procedures All available in- 
Iciirauon concerning the power or pow<r efficiency of each test will 
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be presented The tests, m aimost all cases, will be compared with 
tests based on normal models Unfortunately, the power and the po%ver 
efficiency of a test cannot be determined unless some definite functional 
form for the underlying distributions is specified 

For several reasons the tests will be compared entirely on the 
basis of underlying normal distnbution— that is, the power efficiencies 
of nonparametnc tests relative to the corresponding normal tests will 
be given for those cases where the normal tests are strictly applicable 
We shall be ans\\ering the question much do we lose by using 

these nonparametnc statistics when we could use normal statistics^” 
“How much could our apprehensions cost us if our fear (of non- 
normahty) was unjustified^ ’ 

The more pressing question, “How important is it that mv results 
2pP^y generally rather than to populations distributed normally^” 
niust, of course, remain unanswered, since it can only be ansivered 
by the research worker each time he considers a test of experimental 
data 

The cxpenmental tests presented will fall into two natural but not 
mutually exclusive groups which for con\enience will be labeled tests 
0 / location and tests of relation These correspond, respectively, to the 
ordinary tests of mean difference and tests of correlation, with tests 
analogous to the analysis of variance occup>jng some middle category 


Tests of Location 

THE SIGN TEST This test is useful in cases where the t test of 
differences is ordinarily used, that is, where a set of paired observa- 
tions IS available A common application is the before after type of 
Experiment, in which measurements are made on each subject some 
treatment or stimulus is applied to each, and the measurements are 
repeated The null hypothesis Ho would be that there is no effect, 
'vhereas the alternatives could be either Hi the effect is positive 
(negative), or Hi there is an effect For the purposes of the sign test, 
these are reformulated in this way 

Hq The median of the distribution of differences betiveen before 
3nd after measures is zero against Hi this median is positn e (nccative), 
or Hi this median vs not zero 

The power efficiency (in the remainder of this chapter power 
efficiencies are quoted for 5 percent level tests) of this test in normal 
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samples IS rather low, ranging from 63 7 percent for larger samples 
up to 68 or 70 percent for small samples The sign test with 18 pairs 
of observations is approximately equivalent to a I test with 12 paire 
when the ( test is applicable, 40 pairs for the sign test compared with 
28 or 29 for the t test 

Let us suppose that a questionnaire about attitudes toward 
Negroes has been administered to a group of 22 subjects, after which 
the subjects undertook an extensive study of race prejudice The follow- 
mg results were obtained on a follow-up questionnaire 


Subject 

After 

Before 

Dijfererue 

1 

35 

31 

+ 4 

2 

36 

29 

+ 7 

3 

29 

35 

- 6 

4 

34 

32 

+ 2 

5 

33 

29 

+ 4 

6 

28 

33 

- 5 

7 

33 

30 

+ 3 

8 

28 

38 

“10 

9 

28 

35 

“ 7 

10 

25 

22 

-1- 3 

11 

33 

31 

-h 2 


Subject 

After 

Before 

Difference 

12 

28 

27 

+ 1 

13 

27 

26 

+ 1 

14 

27 

25 

+ 2 

15 

33 

32 

+ 1 

16 

42 

40 

•f 2 

17 

19 

18 

+ 1 

18 

37 

36 

+ 1 

19 

40 

40 

0 

20 

32 

31 

+ 1 

21 

31 

27 

+ 4 

22 

30 

29 

+ 1 


If the null hypothesis were true— that is, if these 22 differences 
were drawn from populations with a median of zero— we should expect 
about half of them to be positive and half negative We see that, in fact, 
17 are positive, 4 negative, and one is zero We should expect that the 
distribution of +’s and — ’s would be about the same as that of heads 
and tails in tossing 22 unbiased coins The probability of 17 heads and 
4 isih ecvn r<^}ietg oi ts less than OGl, set wu wutrAf rejeef 

Ho at the 1 -percent level if our alternative hypotheses are specified by 
HI Had we decided be/orehand to use the sign test with the one tailed 
alternatives Hi the effect is positive, we would have rejected Ho at 
the 1/5 ‘percent level 

The t test of differences of either the one-tailed or two-tailed form 
here would have yielded a quite insignificant result {< 2 idf = 0 60) which 
would have ignored the fact that although the differences were small, 
they were almost all in the same direction 

Probability levels for the sign tot are tabulated in Dixon (1) The 
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5-percent level of the two-t ailed tes t can be obtained from the formula 
[(^ — 1) / 2] — (0 98) -v/fV 4- 1, the level being the integral part of 
this value, where N is the number of pairs in the sample, ignoring all 
pairs with a difference of zero In our sample, JV = 21, the formula 
yields (20/2) - (0 98) V22 = 10 - 4 596 = 5 404 and we shall 
reject the null hypothesis at the 5-perccnt level if there are 5 or fewer 
plus or minus signs among the 21 differences 

The sign test is perhaps the simplest of all distnbution-free tests 
and at the same time is most readily generalized to analysis of variance- 
type problems This generalization will be discussed after the presenta- 
tion of a number of two-sample tests 

THE WALD -WOLFO WITZ RUN TEST (16) The Wald-Wolfowitz Run 
test IS specifically designed to test the null hypothesis Ho two samples 
were drawn from populations having the same continuous distribution 
The alternatives are the extremely large class characterized by Hi two 
samples were drawn from dxferent continuous distributions This test, 
then, would tend to reject the null hypothesis if two populations were 
different m any respect 

The assumption of continuity is a cntica] assumption here It 
implies that gross measurement is the only reason for two observations 
to have the same value 

The power efficiency of this test is not known However, a small 
number of empirical tests by the author have indicated a number 
somewhere m the neighborhood of 75 percent in normal samples when 
the distnbutions differed only in means and when sample sizes were 
about 20 

Let us say that we have two groups of high school students m a 
^arge City, one of which has just completed a course in religious educa- 
tion, the other group not having had this course We wish to test the 
hypothesis that our rehgious-education course had no effect on attitude 
toward religion as measured by a new scale Say, also, that the scores 
On the attitude scale are as follows 

Riltgtous education 52, 53, 71, 86, 95, 108, 115, 120, 141, 152, 
165, 218 

Ho religious education 30, 45, 54, 74, 75, 81, 101, 104, 146, 151, 
170, 171 

The test consists in arranging all the scores of both groups in 
Order and then counting the number of runs of scores coming from 
each group In the example we have^, 45, 52, 53, 54, 71, 74, 75, 81, 86, 
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95, 107, 104, 108, 115, 120, 141, 146, 151, 152, 165, 170, 171, 218 Our 
tes’t statistic is the number of mns. r, which in this case is 12 Under the 
null hypothesis, ro is given by 

r. = £ (r) = (2 NiN,)/iNi + JVs) + 1 


The variance of r, 


2 NiNi {INiNt - iVi - Ni) 


where A^i is the 


number m one sample, N 2 the number m the other 
In the example N; — Ni =■ 12, 

. MM + 1 = 13 

, _ 2 (12) (12) (264) ^ 

(24)= (23) 

», = 2 40 

For values of and N\ larger than 10, r is approKicoately normally 

distributed and large values of r tend to confirm the null hypothesis 
Hence we form the normal deviate C = (r — ro)/»f and reject the 
null hypothesis at the 95 percent level whenever (r — fo)/(r, < — 1 645 
In this example, C «= —1/2 40 = —0 417 and the null hypothesis 
tsould not be rejected Our conclusion would be that we have no 
evidence that such a course has any effect on attitudes toward religion 
Had we assumed normality and used the two tailed t test for 
significance of difference between means, we should have reached 
about the same conclusion (t = 0 73), except that it would have been 
qualified b> the clause “if scores on this scale are normally distributed ” 
It IS important to note again, that, although we used a normal 
deviate to find the probability of our observation, the validity of the 
test depends m no way on the normality of the observations but only 
on the distribution of r in two samples from the same population 
That this distribution becomes normal for large samples can he proved 
mathematically We use it here as an approximation to the true 
distribution, which is much mote difficult to use For sample sizes of 12 
and 12, the approximation would call 9 runs significant at the 5- 
percent lev cl, vv hcreas the true distribution would call 9 runs significant 
at the 6 9 percent lev el The correspondence would be even closer for 
larger samples * 


iFor exact % gn ficance levels for Jtfj, W* 10, see Swed and Evsenhart (14) 
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Theoretically, if the original population is continuous, there 
should be no two observations \vith the same value Because of coarse 
measurement, however, such ties do occur When they occur, the 
sequence of ranked observations is not unique Any of several sequences 
may adequately describe the data, and there is no logical way of 
distinguishing between them Of course, if all ties are within one sample, 
the number of runs is the same for each sequence and our test is 
unaffected, but if observations in one sample are tied with observations 
m the other, the number of runs cannot be uniquely determined from 
the data 

Let us consider our onginal example, slightly altered^ so that there 
IS one tie across samples The combined, ordered sample might be 30^ 
45, 52, 52, 53, 54, 71, 14, 75, 81, 86, 95, 101, 104, 108, 115, 120, 141, 146, 
152, 165, no, 171, 218 Here r = 12 or 14, depending upon which 
observation of 52 is m reality the smaller Neither value would be 
Significant, and consequently our decision would be the same If the 
proportion of ties is very large, ordinarily the number of runs is very 
indeterminate, often ranging from very significant to very insignificant 
numbers, and m such cases this test is inapplicable 

THE MANN-^VH^TNEY OR WILCOXEN TEST (6, 18) Although SUbjCCt 
to the same ambiguities when ties m rank occur in the data, the Mann- 
Whitney or Wilcoxen test is valuable since it provides a one-tailcd set 
of alternatives The null hypothesis Ho is again that the two samples 
are drawn randomly from populations F and G having the same 
distnbuiion Since this, like all nonparametne tests, is a test of distribu- 
tions, the alternatives must be stated rather differently The statement 
of the alternatives is Hi F is stochastically larger than G if A is one 
observation from the distribution Fand Tis from the distribution G, F 
stochastically larger than G if the probabihiv of A larger than Tis 
Sfcater than Loosely speaking, this implies that the “bulk” of Fis 
farther lo ihe right than the “bulk” of G, or that F accumulates more 
‘slowly” from the left tlian docs G If the two populations arc normal 
''tth the same variance, this implies that the mean of F is larger than 
the mean of C 

The Mann-Whitncy test makes exactly the assumptions of the 
^^ald-Wolfowitz test 

dcr \^anrt (15) has investigated the power function of the 
^lann-Whitney test for norm'll samples and has found that, although 
the I test IS more powerful, the differences m power arc quite small for 

’ \n observation of 52 vsas added to the “^o religious education ’ group 
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small samples and not very great even for larger samples This test is 
probably the simplest and most powerful nonparametric test yet de- 
vised for detecting differences m location The Wald-Wolfowitz test 
seems to be better for detecting differences in dispersion 

As an illustration of the use of this test, suppose that from our 
previous example our alternaiwe hypotheses had been H\ students 
who have taken religious education have higher scores on this attitude 
scale than do other students Our statistic U is given by the number of 
students without religious education who have higher scores than a 
specific student with the education This number is then summed over 
all students with religious education Let us tabulate this statistic for 
our example 

TABLt I 

Number oj non rehgtous-educalton 
students higher 
0 
2 
2 
4 
4 
4 
4 
6 
6 
9 
10 
10 

[/- 61 

IS the tilth hypolhesis is Inie, the expected value of CT, is 14 A'aV,, 
or 72 

The variance is = 1/12 IV, A', (|V, + AT, + 1) or 300 When 
the two samples are both larger than 10. (Cl - t/o)/v„ is approximately 
normally distnbuted, and we shall reject the null hypothesis at the 
5-pcrcent lev eht this IS less than -1 645 Inourease,C= -11/(10V^ 
= -0 635, which would have been significant at the 26 percent level 
Ties in rank have less effect on (/ than on r, but large numbers 
of tics still vitiate the value of the test When ties do occur, they should 
be counted as point 


Score on altitude scale Jar 
religious-education students 
218 
165 
152 
141 
120 
115 
108 
95 
86 
71 
53 
52 
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T... m- “isr/Sdi".; 

exactly the same purpose “ -te unimportant in its apphca- 

important feature that ties i applied to grouped data 

tion. Also important is the f Jed on the same continuum, 

for two samples if each samp _ survey researcher, 

It is to be expected that th'S Lperimenter. 

whereas the preceding test i test for extremely large 

The power efficiency of the J known vari- 

samples has been investigated or no efficiency of the 

ances. Preliminary '"‘’"ter of cllss intervals is larger than 5. 

test is larger than 0.90 if t e num interesting application 

Marshall's original paper (7) conta The data 

which should be read as we as f .u- 1037 American Council 

presented here are taken size is extremely large, but 

Psychological Examination. The 5"™? , ^ain at least a total 

us; of the test requires only that mtejval com 
of 10 observations in the two samp distribution of scores on the 

be H.: males and nJlt hypo^ses H, males have 

1937 ACPE, against the cla^of alter t y Table II. 

higher scores than females. The original data 


TABLE 


Scores 
0-29 
30-59 
60-89 
90-119 
120-149 
150-179 
180-209 
210-239 
240-269 
270-299 
300-329 
330-359 
360 -. , . 


Male 

32 

583 

2,061 

4,164 

5,945 

6,536 

5,452 

3,819 

2,227 

1,116 

433 

123 

9 

32,500 


pemalt 

27 

387 

1,606 

3.439 
5,090 
5,376 

4.440 
3,153 
1,832 

789 

248 

60 

3 

26,450 


Total 

59 

970 

3,667 

7,603 

11,035 

11,912 

9,892 

6,972 

4,059 

1,905 

681 

183 

12 

58,950 


U • nal table is then cumulated in Table III 
Each column of the orjgm ^ 
to provide the data from which 
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The final five “" “s™sXe''cumulated percentage 

Cumulated Total column ‘ q 0796 = 4696/58,950), 

ot the total (for example, tn the third row 0 0 / 76 

UT 5, ,s the sum of the two entries immedi- 

g, IS one minus p^, each 9* 

, ‘y 5, and the headings on the last two 

ately below It m columns S 91 a ? *e sum of the cumulated 

columns are self-explanatory Where . number m the 

and n the number in its samp > \ 

S = (r/n) -* 


The variance of S is 

c] * [(l/m) + 




)-t i-' . . 

. + 2i- S /’•?>= 

‘ ' ,1 fc-»+l 

, - • 

wherej is the 050/26,450) - (255,747/32 50^ 

Our test statistic here is 5 - (2®-', ' , ^ s = [(1 /32,500) -t- 

= 7 9036 - 7 8691 = 0 0545 The vanance^o S^^^^ K 

(1/26 450)1 D ^^Vind 1 3263 are the sums of the las. and 
sample sizes, and 1 Obit an 

next-to-last columns _ nnO025611, and <r. * 0 0160 

, 1= (0 68576 X 10-*) (3 73f ) = » OM" the normal deviate 

S IS approximately normally ^ significance level ob- 

in this case is C = 0 0345/0 0160 = 2 .s 0 016, 

tamed from the upper tail of the normal curve 

between the 1- and 2-percent Icve based upon the same 

SMIRNOV TEST (8) The Smirn simpler computa- 

reasoning as the Marshall test t ,s valid for large 

tionally, tests exactly the same VP ’ ^ Marshall test un ess 

samples only Its use is not recommended 

time IS an important factor of the Smirnov test, “‘^'=P* 

Little IS known of the power effi V Marshall test, 

that It IS almost certainly less , ghi-square goodness-o - t 

It IS believed to be more powerful than 

tests (8) m Table HI to illustrate 

We shall use the same data con will be the sample 

the application of this test ^ulativc percentages for eac 

sizes, 32,500 and 26,450, and the 
sample as contained in Table IV 
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Cumulated 

Cumulated 



percentage. 

percentage, 


Scores 

female 

male 

Dtjferetue 

0-29 

0 00102 

0 00098 

0 00004 

30-59 

0 01565 

0 01892 

-0 00327 

60-89 

0 07637 

0 08234 

-0 00597 

90-119 

0 20639 

0 21045 

-000406 

120-149 

0 39883 

0 39338 

000545 

150-179 

0 60207 

0 59449 

0 00758 

180-209 

0 76994 

0 76224 

0 00770 

210-239 

0 88915 

0 87975 

0 00940 

240-269 

0 95841 

0 94827 

0 01014 

270-299 

0 98824 

0 98261 

0 00503 

300-329 

0 99761 

0 99593 

0 00168 

330-359 

0 99988 

0 99972 

0 00D16 

360- 

1 00000 

1 00000 

0 00000 


Our test statistic is the maximum difference of cumulatue per- 
centages, which IS 0 01014 in this cas e, multiplied by a factor which 
takes into account the sample size, Vnm/{n + m), which m our case 
18 120 75 The product of these, X, is 1 2244 The probability lc\cl is 
c-®n OT approximately 0 05 In general, the 5-perccnt level of X is 
1 224, the l-percent level is I 517 The significance level must be 
interpreted somewhat differently here, since it is in reality an upper 
bound to the true significance level Had we computed X using the 
largest difference from all possible classiffcations, w e should have found 
the true significance level which would, of course, have been at least 
as small as the one we obtained here 

It is rather interesting to note that the normal test of diffciencc 
between means, which is probably applicable to these extreme^ large 
samples yields a difference in means of 1 033 with a standard error of 
0 474, which is significant at about the 1 5-percenl level for a one tailed 
test The correspondence with the Marshall test is amazingly close 

PITMANS RANDOMIZATION TESTS (11) Pitman’s Randomization 
tests are a set of nonparametric tests which are applicable in a wide 
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number of situations but difficult for ex- 

computationally to be used often The tests 

tremcly small samples (n < 1 ) responses of two 

Let us suppose as - much time per week 

groups of five people each t ** „ 5 >i We wish to test the hypoth- 

should a man spend doing home repa of hypotheses 

esis of no mean difference against ,he Group 2 

that the Group 1 population has a different m 
population Let the data be as o ows 


Person 

m 

(2) 

(3) 

(4) 

(5) 
^1 


Croup 1 
15 20 
2010 
7 95 
10 80 
6 85 

= 12 18 f 53 


Group 2 
14 45 
6 30 
10 90 
8 10 
8 40 


Person 

( 6 ) 

(7) 

( 8 ) 

(9) 

( 10 ) 

.2 55 


-all ten observations are from a 
Now If the null „„dy a 

common population, and the p B ^ carrying out 

matter of chance—* e » any of the ^ 5 ^ alternative 

such a split IS equally likely ^ould be more than a chance 

hypothesis IS true, we expect that the ^52 such experi- 

difference between the group mea nUgervations in one group an 
meats should we expect the five arg ^ thing occurred, we 

the five smallest in the other 8*^^^ 4-T3ercent level Similarly, 
should reject the null hypothesis a among the twelve largest 

the observed mean difference wou times if the observa- 

possible such differences less *r.o,;-.r. if the grouping is «ndom 
tions are from a single " ..mnt to find 12 groupings of the 

Here, then, is our test We simp y a „ pn n difference than the one 
10 observations which give a amone the twelve largest, we 

(2 55) we found If the one we found is arnong 
reject the null hypothesis at the 5-percent level 

It , the null hypothesis More than 

In our example, we cannot J 
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twelve ccmb.nat.ons which give a larger mean difference are shown 
below 


Persons in Group 7 

X. 


1, 2, 6, 8, 4 

14 2 

6 59 

1, 2, 6, 8, 10 

13 72 

5 63 

1, 2, 6, 8. 9 

13 66 

5 51 

1,2, 6, 8,3 

13 63 

5 45 

1, 2, 6, 8, 5 

13 41 

5 01 

1, 2. 6, 8, 7 

13 30 

4 79 

1,2, 6, 4, 10 

13 70 

5 59 

1,2, 6, 4,9 

13 69 

5 57 

1, 2. 6, 4, 3 

1361 

5 41 

1, 2. 6, 4, 5 

13 39 

4 97 

1. 2, 6, 4, 7 

13 28 

4 75 

1,2, (5, 9,10 

13 25 

4 69 

1, 2, 6, 10. 3 

13 22 

4 63 


Lnder rather general conditions, this test leads to the same con- 
clusions as would the ordinary t test, but the power efficiency of the 
Pitman tests m nonnormal universes is unknown 

THE MEDIAN TEST (17) The median test is rather similar to the 
run test presented eatUer and is a generalization of the sign test. It is 
subject to somewhat less difficulty with ties in rank than is the run test, 
since ties are important only when they occur at the median of the 
combined sample It has the further advantage of being rather in- 
sensitive to differences in dispersion- If, for example, the ordered 
samples A and B took the form aaaabbbbbbbbaaaa, where a is an observa- 
tion from the A sample and b an observation from the B sample, the 
run test would reject the null hypothesis of no difference in the two 
populations, since there arc only three runs Since the samples have 
the same median, the median test would not be significant The dif- 
ference m the two samples is the wide dispersion of A compared to the 
dispersion of B The hypotheses tested here are Htj the two populations 
have the same median, against Hx the medians are different 

Results on the power efficiency of this test are not available As 
was noted above, it should be more powerful against alternatives which 



Distribution-free Methods 559 


». - p— -:t « »• 

have a, observations in one sample a • observa- 

median of the (ni + ni) observation populations have 

tions in each sample above this me i j about ni/2 and 

the same median, we would expect as a criterion 

a,/2 Deviations from this expectanon ^^^^^^^^^ 

If we call the observed number of tabulation 

each sample an and a.,, w e need only compute x.a 

/. 1 


mi 

m — mi 


mi 

m — mj 
ni 


(m + 
(m + 
ni + ”2 


Let us take as an n *rfo”llowing manner on a 

10 are male and 20 female Th y 
short attitude questionnaire 

Afra 12. 21, 28, 37 f > f 30, 31, 32, 33. 34. 
Wcmm 12, 13, 15, 15. 21, 23. 24, 24, 

34,36,38,43,44 ,, ,1 5 _, e , 15 observations 

The median of the combined =amp^ > ^ 
are smaller, 15 larger The chi square 


Larger 

Smaller 


M(n 

7 

3 

10 


Womtn 

8 

12 

20 


15 

15 

30 


1 C 1 = 2 40, with probability level 

Herex?,, = [30(60)i]/[10 20 *5 '5] ytelds a 

about 12 percent The I test of differ 5l,ould conclude 

significance level of about 8 '' between men and women 

that we have no evidence of any difference ^ 

On this questionnaire can also be ana yze 

The example used to illustrate the run 
here, vielding a table 
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12 

12 

24 


7 5 

5 7 

12 12 

j — 0 67, which IS not significant even at the 50-perccnt level 
When the combined sample contains an even number of observa- 
tions with no tied observations at the median, there will be (m + nj)/2 
observations larger than the median and the same number smaller 
than the median When (nj + n*) is odd, the sample median is one of 
the observations and it is not determined in which cell that observation 
IS to be entered If we make the rule arbitrarily to include it in the 
class less than the median, the test in the long run will not be affected 
The same arbitrary rule provides us with an approximate solution 
when lies occur at the median, that is, to call mi and mj the number of 
observations above the median, and ni — mi and m — m* the number 
of observations less than or equal to the median It is only approximate 
because our measurement is too crude to differentiate between observa- 
tional values 

AN ANALOGUE OF THE ONE FACTOR ANALYSIS OF VARIANCE TEST* (9) 
This test IS a straightforward extension of the median test m the same 
way as the simple F test is an extension of the t test We go from testing 
the null hypothesis that two medians are equal to the null hypothesis 
that a number, k, of medians are equal, against the alternatives H, 
that at least one of the k medians is different from the others 

No information is available concerning the power efficiency of this 
test Empirical studies comparing this test with the F test m normal 
populations have yielded quite good results 

Let us say that we have k different levels of the factor with n, 
observations at the ith level We find the median of all the observations, 
count the number of observations at each level above the median, the 
m.’s, and enter these in the first row of a 2 by A table In the second row, 
we enter the number n, — m, The x* test of independence of rows 
from columns is then a test of significance of differences among medians 
As an example, we shall analyze the following data from Edwards 

( 2 ) 


»Vanoiis other nonparametric analyses of specific experimental designs 
discussed fully m Mood (9) 


are 
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Levels 


1 2 


13 7 

9 4 

8 4 

7 1 

8 10 

6 7 

6 5 

7 9 

6 5 

10 8 


3 

4 

5 

12 

10 

13 

11 

12 

6 

4 

9 

14 

9 

7 

12 

5 

15 

13 

10 

14 

10 

2 

10 

8 

8 

17 

4 

3 

14 

9 

6 

12 

11 


Mean ^8 6 

Med. [S) 7.5 6 


7 12 

7 12 


10 X = 8.6 

10.5 Grand nied. W = 8.5 


An F test of these data yields a highly significant 


result (F4,45 — 


6.52). Our test is as follows: ^ 

1. The median of the 50 values is B-t). 

2. The 2 by 5 table is 


Above 8.5 
Belo%v 8.5 


1 

3 

7 

10 


Leiel 

2 3 

2 4 

8 6 

10 10 


4 

9 

1 

10 


5 

7 

3 

10 


25 

25 

50 


3 (5-3?_^(5r:V + (L^+--- + ^^ 

Xid f. — ^ 1 ^ 5 

_ the value of F, still 

4. This value, though not as signitican expected to give 

attains the 1-percent level. The test would not 
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lafons A further reason for our ^ ,est Us.ng the 

that we have used only an approxim ^ r = 14 97, which 

maximum likelihood value (9, p 2 ) w 

attains signiheanee at the '/.-percent level 

nificance level of F We ^^ou rejec , populations with the 

hypothesis that the 5 samples were drawn from popul 

same median 


Tests of Relaltonshtp . of 

This seetion will be devoted to a m this 

association in a bivariate population power efficiency They 

section has been adequately investigate j assumptions are 

remam useful, howeser, especially when normality 

intoleiable , „ . oh the vanous tests 

One example (13) will be o owe estimation The 

discussed, and also through the fo ot jyjg,toward-children scale 
data consist of scores on a gn educational program 

obtained by eighteen couples before an scores obtained 

Five-point scoring was used, an t ' j being 0 to 120 

ranged from 36 to 110, the maximum ^ ^ 

A high score indicates a “permissive amtude, 

amtude The data are contained m a e test is the test 

THE coNTihGENCY TEST (9) The on the other, fitted 

of whether the linear regression line o -phe test becomes just 

nonparametrically (9, p 408), has ® ble formed by the .V median 

a test of linear independence m the one would expect about 

and the Y median Under the null hypo . indicate associa- 

n/4 observations m each cell Departu associated wit 

hon-.,, high values °f positise (negative) assoe.a- 

Ingh (lo^^ ) X alues of the other This is can \ 

nn«ible on the data m our 

tour different tests of interest are association between 

example Tlie first test will be, “Is there 2X2 table shows 

husbands’ and w ives’ scores on the is not significant at the 

wmc positive association, but with a x ’ frequencies are not 

10-peixent level It should be noted 
lirge enough to make the clii-squ ire 
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Husband 


Wife 


>med 


'>mtd <med 

6 3 


9 


<incd 


9 

18 


The same test on the “after” questionnaire can be made on the 
2X2 table with the same results as before 


Men 


Women 


>me<l 

<med 


>med <med 

6 3 

3 6 

9 9 


9 

9 

18 


The other tests, women before and after and men before and after, 
show highly significant association, as one would expect The phi 
coefficient can be used as an estimate or index of association, being 
+0 33 in these cases 

We should not expect this test to be very powerful, since only a 
few slight shifts in scores near the median could radically alter the 
conclusions drawn Its ease, however, recommends it as a simple 
preliminary device to find highly related variables 

THE CORNER TEST (tO) Th\s tcst lays much more emphasis on 
the association observed between the two variables at the extreme 
values of each than does any similar test so far proposed In application 
It is not quite so simple as the one above, but it would seem to be much 
more powerful This emphasis on extreme values is a rather valuable 
contribution of the test, since one is often most interested in the 
charactenstics of persons who deviate a great deal from the norm 
This interest is often due to the feeling that minor extraneous factors 
may disturb the relationship between two variates where neither is 
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of the attitude scale mentioned prevrousiy. The scores 

Figure 6. , , ... .i.. medians. > and x The 

A scatter diagram is used f 

tour additional boundary lines are added 



f.o. G. An illustrauon of the use ot the corner test on .he data r,t 
Table V. 


1 . Li is drawn parallel to ) and as ^ “on 

condition that all points above I.t ate on be abos e 

side ot u. It Li were closer to ), a point o ,be “on 

it. It the first point on the “olT’ side is tied wdh P 
tide, the boundary line is drawn „„ one s.de of.y, those 

2. Similarly, all points to the right o i „„ ono side 

Wow L, are on one side of JT, and points to th 


of }. 
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3 Points above h are counted, and a plus or minus sign is 
attached according as the points are to the nght or left of x In out 
illustration, this number is 6 

4 Points to the right of Lj are counted and a plus or minus sign 
IS attached according as they are above or below In our illustration, 
this number is 3 

5 Points below Lz are counted and a plus or minus sign is attached 
according as whether the points are to the left or right of X In our 
illustration, this is 1 5, counting only half of the tied point 

6 Points to the left of L* are counted and a plus or minus sign is 
attached according as they are below or above In our illustration, 
this IS 3 

7 These numbers are algebraically summed to get the test 
criterion (in our illustration, r = 13 5) 

8 In the limiting case, as sample size becomes large, the 5-percent 
significance level is given by |rj > 11, and this criterion is adequate 
if sample size is larger than 10 For sample size of 11, not all points 
need to be outside the boundary, since “corner” points are counted 
twice In our case, r — 13 5, so the null hypothesis is rejected at the 
5 percent level 

It should be noted that, both in this test and the preceding one, 
the procedure cannot be followed without modification if there are tics 
either at one of the medians or on a boundary line, or if there is an odd 
number of observations 

When there is an odd number of observations, one observation 
will he on each median For example, the median of x values might be 
71 and the median of values 84, with the two observations being 
(71, 19) and (47, 84), and we could not tell whether either point was 
on the on ’ or “off” side of the median The ambiguity can be re 
moved by substituting the observation (47, 19) for the two observations 
This substitution does not bias the test m any way In this way we 
have an even number of observauons vathout affecting either median 
It is possible, of course, that the odd observation hes on both medians 
In our example it might be (71, 84) In this case we may neglect this 
observation, since it cannot affect the quadrant sums 

Tics not on a median are important only if they affect the position 
of one of the boundary lines, L In Figure 6, the two observations 
(61, 43) and (81, 43) arc tied on Z,j Consequently, we do not know 
whether to count (61, 43) as outside or inside the boundary The 
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authors of the test suggest that ^ 
te\t:arr:ie'rto co"ult the uutnber of obsersattons t.ed on the 

If m our example, 5 points had lam on / * 

nght, the 5 points Mould statistics is perhaps the 

RANK CORRELATION (5) cfatistics Onginall^ con 

oldest and best kno^vn of all a product moment 

ceived of as a shorthand method o tnany assump- 

correlation, a conception whic ““ j orders of mathematical 

tions rarely satisfied in practice, p tis to be considered as 

approximation, they have come more and more 

indices of association /J?r w nooulations except that 

No assumption need be made order-i ^ , that a 

samples of observations can be a g ^ ^ the sample, 

sunple ordering exists, and even where there are 
reasonable approximations can , earliest and best kno\Nn 

The Spearman coefficient p fo) ^ minus one and is 

of rank order statistics It ranges ,5 closely related 

quite simple to compute This, an reasons ive have for includ- 

to the concordance coefficient, arc t e o . js m all other 'n ays 

mg p. since Kendall’s r (5), to be d.scu^ed _ „)]. 

preferable The computing formula is p difference be^s ecn 

"here n is the number of objects ran c a other For example, 

the object’s rank on one ranking an its r ranked by tis o judges 

suppose four attitude items on anti ^ , , jf he endorsed cac 

that IS, ho\\ much anti Semitism one '' 

Item Suppose the rankings \^c^c 

A 


Item 
Judge 1 
Judge 2 


Here f 


Juage^ . M = 

. , (9 _ 4)’ + (-l - 
4 =s (3 — 3)’ + U ” ^ ' 

H, and p =’ I 1 [f4l/[(l/6) (60)1 “ more difficult Tics 

\S'hcn tics occur, the process bcco 
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tend to raise, spuriously , the computed correlation between rankings, 
and a correction must be made We can illustrate this case by supposing 
that Judge 1 had been unable to distinguish between Items 2 and 3 
Tlie rankings would h-iNe been 

Item 12 3 4 

Judge 1 1 2 5 2 5 4 

Judge 2 3 2 4 1 

and Stf* =15 5 In the denominator wre would replace (1/6) (n* — n) 
by \/(l/6) (n“ - n) - 14 V(l/6) (n» - n) = \/^ = 9 75, and p 
would become 1 — 15 5/9 75 = —0 41 The correction for ties in gen- 
eral IS that we divide bv "y/ (1 /6^ (n* — n) — T \/ (1 /6) — n) — V 

instead of (1 /6) (n’ — n) where T = 1 /12 (2 (i* — /)] and t is the num 
her ofobjectsinvoKed in one tied ranking, and U — 1/12 [2 (u® — «)], 
u being the number involved in the other In the ranking 1, 3, 3, 
3, 5, 6 5, 6 5, 9, 9, 9, for example, T = (1/12) [(3» - 3) + (2® - 2)+ 
(3* — 3)] es (1/12) [54] = 4 5 If the other ranking were 3, 3, 3, 3, 
3, 6, 7 5, 7 5, 9 10, (/ = (1 '12) [(5® - 5) + (2® - 2)] = 10 5 The 
correlation would be 1 - ((12 5)/V(160 5) (154 5)] = 0 921 

In the altitude study example, the Spearman correlation between 
husband s and wife s scores on the first administration is 0 586, on the 
second administration, 0 649 Couples seem to have become more alike 
following the training course, although we have no way of testing this 
The correlation between the rankings of men before and after training 
is 0 765, the same coefiicient for women is 0 804 

3Nhcn n is larger than 9, wc can test the hypothesis that p = 0, 
against the alternatives p 0, by forming I = p"\/{n — 2)/(l — p*) 
which IS distributed as Student’s I with (n — 2) degrees of freedom 
For the correlation between husbands and wives on the first adminis- 
tration, i = 0 586V16/11 - (0 586)®] = 2 89 with 16 degrees of 
freedom, which is not quite significant at the 1-pcrcent level For n 
smaller than 9, tables in Kendall (5) give significance levels 

\\ hen a number of rankings of the same objects exist, it is some- 
times of interest to determine how much agreement there is among the 
rankings lor example, wc might have a number of judges ranking a 
sci of staiemcnis for pro war ftcling and we might wish to use average 
rankings to get a best estimate of some ‘'true ranking,’ with which all 
judges more or less agree Wc would want to be assured that there 
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. , that the ludges «ere not ranking the 
exists some such ranking . roefficient, W, is a measure 

statements at random The c;efficient PT = 1 0 

of the amount of agreement animg J “S p„s3,bie 

,f all judges agree perfectly, zero if 

The coefficient is given by the form 

W = 12S/[m’ ("’ "■ 

where there are n objects being ■* around a mean of 

the squares of the rank sums or e consider the two 

[a. („ + l)]/2 In our attitude-scale ,be wife as four 

test scores of the husband and the two of “objects” being 

rankings by “judges” of the*thenth<=^^^^^^^^^^ 

ranked, the number of couples, «,,idee” is 9 5» hence the 

four The mean rank assigned e^cn j| s first 

average rank sum is 4 (9 5), or 38 ,oople (4 -H6 5 

couples (16 + 12+14 + 18) so o. The sum of 

+ 4 + 14) = 38 5, for the coup! > __ + 

squares, 5, is then (60 - 38)^ + (^ 5693 5/7752 

In this case n = 18, m - 4, and w * 0 against the jy®®) 

= 0 734 To test the null hypothesiH. [(„ _ 1) m/ 

W > 0, we can use the Snedecor (12) ,^_es of freedom m the 

(1 - Ht) = 8 35 with [n - 1 ] degrees of freedom m the 

numerator and [(m — 1) (a ^ ^ hiehly significant amou 

denominator, here 16 5 and 49 5 There ordering the couples with 

of agreement and we can proceed with ordering 

respect to “permissiveness ” „„.i,„on between the 6 rankings 

The average Spearman _ j), and m our example 

IS given by the formula p»t “ 

° 844 .^„,„d tor the same purpmc as 

Kendall’s tau (5) coefficient -=^”®gJ°ome«hat more difficult 
the rho coefficient mentioned earlier reneralizable into a partia 

to compute, it has the advantages o ^ normal distribution or 

corrclauon cocfficeut, and of having an almost 
samples as small as 9 __ o? n (n I)» again 

The computing formula is t ■! /^j-mmed m this manner 
tlic number of objects ranked ana . ^ rankings 

(1) Order the objects according to one o. 

(2) In the other ranking, for each object 
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objects to Its right having a smaller rank and the number having a 
larger rank Subtract the former from the latter 

(3) Sum the numbers thus obtained for all objects to obtain -S’ 
As a single example, consider the rankings 1, 4, 3, 5, 2, 6 and 2, 3, 4, 

1 5, 6 If we arrange the first m order, the order of the second becomes 
2, 5, 4, 3, 1, 6 Start with 2 Since 3, 4, 5, and 6 are to its nght and 
larger, and only 1 is smaller, the contribution is 4 — 1 or “3 ” For 5, 
only one number to its nght is larger, 6, and 3 are smaller, so that 5 
contributes a “ — 2,” the number 4 contributes a “ — 1,” the object 
ranked 3 contributes “0,” and the object ranked 1 contributes a “+1 ” 
These contributions added give 3 — 2 — l+O+l, orl This is the 
value of S' t is (2) (l)/(6) (5) = 1 ^15 
In the case of ties, t becomes 

S 

Vo 5n (n - 1) - r V0 5« (n - 1) - 

For this coefficient T — 0 52/ (/ — 1) and 1/ =» 0 52u {u — 1), where 

1 and u are used m the same way as in Spearman’s coefficient For 
example, in the ranking 1, 2, 4, 4, 4, 6 5, 6 5, 9, 9, 9, the correction is 
0 5 [3 2 + 2 1 + 3 2], or 7 If another judge had ranked the same 
objects 3, 3 3, 4 5, 4 5, 7, 7, 7, 9, 10,Swo uld be 7 + 7 + 7 + 54-5 + 

2 + 2 + 2+1 = 38, V0 5n(n-l) - Twould be V(0 51 (10U9) - 7, 
Vo Snjn -_1) - OSU would be V(0 5) (10) (9) - 7, and r « 
38/ V38 V38 1 , as we might expect, since each ranking contained 
the same number of ties and was in the same order 

If we label the columns of Table V in order 1, 2, 3, and 4, we have 
Tn = 0 457, Ti3 = 0 656, n, = 0 477, tjj = 0 401, tsa — 0 559, and 
T34 = 0 493 

When n IS greater than 10, t is about normally distributed with a 
mean of 0 and a variance of U/9M12 r + 5)7(71^ - n)] In our example, 
=* [2/9] [41/306] = 0 029774 and a standard error, tr, of 0 1725 
The normal deviate for ti» = 0 457/0 1725 = 2 649, which is signifi- 
cant just under the l-percent level and does not differ much from the 
lest of p for the same rankings 

To discuss partial correlation, let us consider the three rankings 
Q and R 

P 12 3 4 5 

a 1 3 4 2 5 

R 2 3 4 1 5 



erf 
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Let us make a table wtth an enhj for Z 

n entered if the pair ts m the order at the column head. 

if It IS reversed 


( 12 ) 

+ 

+ 


(13) 

+ 

+ 


(14) 

+ 

+ 


(23) 

+ 


(24) 

+ 


(25) (34) (35j 


(15) 

+ 

+ 

_ _ + + 

The correlation between Q. and R. partialhng out P, is the phi 
coefficient in the following 2X2 tabulation 

Ranking it 


+ 

+ 

+ 


(45) 

+ + + 

+ + + 

+ + + 



Pairs 

Pairs 



agreeing 

utth P 

disagreeing 

iLith P 

Totals 

Pairs 


3 

8 

agreeing 
\\ith P 




Pairs 

disagreeing 

1 

0 

2 

with P 

Totals 

7 

3 

10 


Kamiinc 

d 


n ■JOT Xhis niorc 

The phi coefficient is -6/V(2r(3)l7rW " 

easily determined from the formula - — ^ 

roNr=[roN-rrorrN]/V[l-^Wn— ] 

Here TQK = 0, Tpa = 0 ® 40^^nd 

r. . = [0 - (0 60) (0 40)1/V(m^^ 

The partial correlation can be which P is rating arc 

ween the ratings (i and i? when the a “judges” arc 

This interpretation h^'^s ^^p^ble of arran, 

lartial taus 1 


Ween the ratings (i and i? when the factor ^ ^ 

''cW constant This interpretation hoios capable of arranging 

actually judges, or tests, or any other ,,u„on of partial taus is 

“hjects in rank order Unfortunately, tne oi have some 

aoi known In the short example jusi co ^ ranking is 

'aidence, however, that when the factor on . p ppgbt be an 
*'ased IS held constant, Q. and R arc negaiii ^ ^ rnusical 

'ntclligcnce test, Q_ a test of number a » »t% , 
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ability In that case one would say that among persons having about 
the same intelligence, those having high number ability tend to have 
lo\ser musical ability than those having low number ability 

An interesting use of this technique with our attitude-scale 
example would be to correlate the ranks of husbands and wives follow- 
ing the training period, partiahng out the correlation due to the ranking 
of the couples before the training To obtain a ranking of the couples 
before the training period, we can rank the sum of raw scores for 
husband and wife before training We can call this ranking “c ” The 
correlation between the women’s scores after training and “a” is 
t «3 = 0 669, while Ta< = 0 623 The correlation between wives and 
husbands after training, partiallmg out the factors common to those 
present before training, is 

t 34 - = [0 493 - (0 669) (0 623)]/V(0 552) (0 612) = 0 133, 
indicating an extremely low relationship This might be interpreted 
as meaning that the relationship between husband’s and wife’s scores 
after training is mainly due to factors present before training If a 
partial correlation of 0 133 is significant for samples of size 18, we 
could also say that training tended to make husbands’ scores and wives’ 
scores more similar with respect to relative ranking in their respective 
groups 


ESTIMATION WITH ORDER STATISTICS 


PercenliUs (1, 9) 

One of the most interesting results from order statistics is that the 
expected proportion of the population falling between two ordered 
observations is l/(n -h 1), where n is the size of the sample—: e , the 
sample is expected to divide the population into (n -f- 1) groups of 
equal size Because of this, the percentile points of the sample are 
estimates of the percentile points of the population For clarity we wil’ 
define as the ilh smallest observation in a sample of size n For 
example, with a sample of 19, the fifth observation from the bottom, 
Xiii} is an estimate of the lower quartilc, the twenty-fifth percentile 
point, and in general, the ilh observation from the bottom, Xi\n is an 
estimate of the 100i/(n -f 1) percentile point For a sample of 200, for 
example, the closest obscrxation to the twenty-fifth percentile is the 
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y which estimates the 24 9th percentile, 
fiftieth from the bottom, Asnisoo "" for the 100th per- 

Kwe require a ^^-P-cent confidence 

centile, we look for the ^ of foe samples of size n we 

AT.!,, so that in L population will he between \,,n and 

draw, the /)th percentile of the pop 

X.,, Any r and s that satisfy the equation 


iO' 


foi (1 _ f,)"- = 0 95 


S- .y, bv the binomial theorem In most 

wiU determine such a Xr\n and A«iis» Y , ^ ^ satisf> the equation 

cases we shall not be able to fin an r a it rather closely 

exactly, but we shall usually be able PP ^ 95-percent upper 

For a sample of size 10, we mig t wis might \\ant a number 

confidence level for the tenth percenti e ’ be larcei than 

which in 95 percent of the samples we choose r equal to zero 

the tenth percentile of the population . is (1/10)“ 

, i-u n 95 The first term is ^ 0 / ^ . 

and sum terms until we reacn u ... .Vir. «malle'i sample 

(9/10)u, or 0 3483. This is the ^f foe population The 

observation is larger than the ten p 7354. 

second term ,s (\«) (1/10)> (9/10)’ = „bservauon is 

This IS the probability that the ^ nopulation Continuing, the 

larger than the tenth percentile ot of the hist three 

third term is (\«) (1/10)’ (9/10)’ = 0 1936, the sum of 

terms is 0 929, and the sum of the *■■■’' less than the lourth 
that the probability that the tenth per ^ probability 

smallest observation in a sample o ^.,on is about 0 99 

that u ,s less than the fifth smallest observ confidence intervals 

When we have large samples and wis n normal .approM- 

for percentiles between the twentiel an 

mation can be used , (,fo percentile u J 

A95-percentconfidencciniervairor th I 1^^^^^ _ ofijoO) 

1 .. <f)lh perc entile < , r/i 96/IOO) ViV^OO - />)! 

V^MiOO - p)] and s IS [/- fn + 1 ) /lOOl + ^,5 ercciit confidence limits 
for example, let us find the approvtmale 1 rsnmaie i 

'or the twentieth percentile in samples of 7 Z ,^^,f imgth 

.e^ tV^>.».vtr^nth observation ro ^ n 784V^* 

file interval is tU96/l00]V7n^^S°^ ' 
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mately 7 Thus, the 95-percent confidence interval for the twentieth 
percentile is to JVsaiTg The interpretation is this “In 95 percent 
of samples of size 79 drawn fiom a continuous population, the twentieth 
percentile of the population will lie between the ninth smallest and the 
twentv-third smallest observations 


Estimation oj the Cumulative Distribution (7) 

A method based on the same theory as the Smirnov lest discussed 
earlier is available for estimating the cumulative distribution (ogive) 
of the population from that of the sample In Figure 7, we have plotted 



F,= 7 A compar,so„ of ,ho dMr.buuoo of p„,ra,„.„g scor« on ,h= 
Stott-Bor>o„ n,u.udn ,calc ».th a normal d.rmbunon bav.ng 
die S4inc mean and variance 
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an example based on the 36 f s the etmult 

scale before the training penod The rent gg gO-percent 

live euwe for the sample - ThatTif - dr^^^ 

confidence bounds for the popula I „„mher of times, ue expect 

size 36 from continuous populations a large will contain, 

that 90 percent of the confidence inte^als so compum 

employ, *•= “ 22 /\/JV where N is the sample size In 

puted by finding - ^he „pper confidence bound is parallel to 

our example, do so - 0 20 ^ ^ q percentage points The 

the sample cumulative and shifted ''““c ^ ^ ^ shifted doun- 

lower bLnd is parallel to the sample enmulat.ve 

Hard 20 percentage points oamule to a theoretical 

If we wish to test goodness^f-fit ^ If, at 

distnbution we need only plot the t “cc reject, at the 

point, It does not he between con^f^^h^thesis H. the sample 

level used m computing the bounds, ,,_,huuon The smooth curve 

was drawn from a population wit t is with mean of 68 22 

IS the cumulative distribution of a --*,relv within the bounds, 

and standard error of 20 125 Since drawn from such 

we cannot reject the hypothesis that t e 
a population 


CONCLUSION 


The relative emphases of this c ap relative importance 

estimation should not be seen as an opmi rg^jj^ation of the magni 
of these U\o phases of statistical tec niq ^ certainly be of prime 

tudes of differences m different population n^ust 

importance for the scientific ^^'mt of view of this chapter 

ascertain where differences exist o P° nniilation differences 


vuucsoiainercnccsinumv.*'-*- r-. _^^j,rnon but nn>i, 
importance for the scientific go®'^ P y^ew of this chapter 

ascertain where differences exist « P° population differences 
has been that the social scientist scarchi g ,ly as well as in 

must r -■ ■■ ™-v,mnllv open mind, statisi ^ i.n-^^ecs and 


fam wucrc — |- oOPUiaii'J** 

een that the social scientist scarchi g as "cll as in 

raust proceed with a maximally open dilfercnccs and 

his field of interest After investigation information about 

«« of the Smirnov or Marshall techniques g njnmation 

■he shape of the population „„.ne the d.sinbuuon 

procedures, based on the parameters w 

«W.- . * 1 .a rvstl^ll Ot 

cur\e, c 


population 7dot=rm.ne the d.sinbuuon 

"■cuurcs, based on the paramctc 
^c, can be used on those problems i^gi much o 

Imcstigations in the biologica ic 
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their data is well fitted by normal theory It reinama an open question 
whether this is true in social science data If it is true in our case, we 
are fortunate, if not, we shall at least know, and be able to act on 
our knowledge 
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PART V 


Zhe 

Kcscatch Titidiaps 


than in many fields, 

In the field ot social concerned with ^ 

ihe research workers theinselves . jj^piobleros of so““' “ ‘ ^ 

application of their findings an ^,etween the social an 
There are two important differen of 

natural sciences in this respect. , .pp^ing of the 


application of their findings •• ‘ ^^^^p the social a.- 

There are two important differen jpjppjs of 

natural sciences in this respect. j.rstanding of the 
discoveries calls for little or no •p.^pputtheapplitntiot'* 
involved by the user once the enginee gcience, however, t e i 
*e form o' a mechanical device. In soc p d with- 

ings having to do with interpersona second differ 

oat a realLderstanding of their "’“"‘^pging human behavior, 
that application in social science matter of our fiel • 

and this is in itself part of the very S 1 jp^plved in tlK “PP , . 

Moreover, the social scientist tccom^^jP^ ^ celationsh.p 
'ions ot his findings because he often 
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Zhe 

j^scutch TitidiHps 


, than in “any 

in the field oE --1 -eatdn concerned with .d-e 

the research workers *®|“^'' 3 nd with social and the 

application oE their fin i ^.g^sences ’^^tw^n t ^^^sal-science 

There are two importan useEulncs principles 

natural sciences in tltj* t“P understanding plication into 

discoveries calls Eor ht engineers have pnt ^^evcr, the find- 
involved by the user once t ' ^ jn social „i,j,. 

the Eorm oE a tn'dtn"^ ejsonal relations ailEcrence is 

mgs having to do with P^^.^ meaning- behavior, 

out a real understanding means ch- S e„r field- 

ihat application in soci- subject ntj* ^ in ,i,c applica- 

and this is in itselE P^tt <>‘ J Uecom« ;"'°';7,o,e relationship 
Moreover, the social scien maintains 

tions ot bis findings bccas 


i 



CHAPTER THIRTEEN 


The Utilization of 
Social Science 


Rensis Likeit and Ronald LippM 


, scientific effort may hate 

The results of any piece of i^teroretatjons emerging 

three kinds of usefulness The findings an . who will 

hom It may be of interest and \alue to o us 

be asking such questions as "How human behavior? 

larther insight into individual nations are presented 

What new methods, hypotheses, an g , ^ more intelligently 
™ this study’’’ "How can I think an p -jneralizations com 

a social scientist because of the new ac ^^ju^ts has as its goa 

‘ig horn this study? ’ Utilization by ® ° .jOcaih valid know 

*he production of a greater body o 

"ige 1 he of value to profes 

The findings and interpretations may business executive, a 

“'“nel workers and citizen leaders such leader, a gov ern 

teacher, a community committee chairrn , leaders have 

™'nt administrator, or a personnel goals are 

'“Ponsibihty for public service and so some 

'he improvement of policy making, P a ,hese leaders vsi 

’"a of social activity In turning to findings mean lor the 

^ asking such questions as ‘ What do f.„ngV ’Will anything 

h'“Mems of planning and acting I tim 8 



with ihe people, the groups, and the communities he studies, to 
ensure their cooperation in his research. 

The following chapter deals with procedures and policies in the 
utilization of research results and the problems which arise in such 
application. 
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situation might be much more satisfying talking n ith some 

different in cltain respects Velm ^ that she 

other officers, the PT A ‘’"T.Lp a meetings Per 

should not be satisfied with 50 P““"““™e,nntent adininistiator 
haps certain clues have convince g generally re 

that there is a great deal of wasted e Perhaps for some 

garded as an important and P™® ,) ,,35 the idea that all 

reason the chairman of a community ntmives of the various 

the competition that goes on between representatives oUhe^ 

community groups who sit on his committee ^ j g^peri 

haps the manager of a textile plant has read the r<=porr ^ P 
ments by Coch and French (9) and is "“'’f P,.te 

hie that the productivity of his plant w ic 

good by indiiry standards could r-PV ^P ;;:;’;;.e„t.alit\ 

In each case this image or tentative q «n«ibilities 
stimulates need for further information ^ ^ accepted the 

Some leaders and organizations n discoveries 

standard that a continuous effort to l^eep p persons nncl 

and to try out new ideas is an impoinnt goal 

groups the utilization of science has d rec'trd to the 

This state of affairs is of course not and oij;anizi 

social sciences but a growing numbei of u 

lions are explicity establishing such a goal [or help this is 

Even when there is motnation to turn J® ^ interpretation 

just the beginning Complex 7i examine these prob 

and application must also be sohed s' desire to apph 

in two types of situations (!) svhee of a pres 

scientific knowledge discovered ehcioneic research prore 

problem and (2) where there is a desire to 

durei directly to help solve the present ^ inal>sis of tht 

this distinction because it stems important arc (picstions 

process of science utilization In the first , ^jone elsewhere 

to whether and to what extent the . there arc ques 

applies to the decisions and actions m question ^ ,^icd to the 

'•ons of how the research from elsewhere ^ l,e rcahsn 

C'ant actors m such a was that its practica problems 


actors m such a was that its practica problems 

assessed nnd acted on In the second case ^ jj,c prob 

? "hether the research is focused on major i collection acti'inc^ 
f^'hcr than on „ni,noi.w whe.I.er ihe dan collec 
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they found help me to do my job more intelligently or efficiently? 
What have the scientists found that will help me right now? 

The findings and interpretations may be of value also to the 
public —I e all citizens as individuals who are living in and adapt 
mg to the complex and changing twentieth century society This 
utilization has the adult education goal of helping every man toward 
an understanding of the dynamics and potentialities of human activ 
ity as a means to the achievement of a more rational and satisfying 
personal and collective life 

In this chapter we are interested primarily in ways in which 
social practitioners and all citizens can utilize the resources of social 
psychology to improve personal insight policy making, program 
planning and individual and group action Other chapters of this 
volume have dealt with the standards and criteria by which scientists 
learn from one another 

The utilization of science will occur only if the person or group 
somehow becomes ready to look for and to use scientific resources 
m the solving of problems This readiness and initiative seem usually 
to depend upon these three sources of motivation (1) problem sen 
sttivity (2) an image of potentiahty, (3) a general experimental 
attitude toward innovation 

Motivation for the use of scientific findings and methods often 
stems simply from the fact that the present state of affairs is unsatis 
factory for someone Perhaps the chairman of the P T A program 
committee finds that attendance at meetings is showing a downward 
trend the business executive discovers that the productivity of his 
plant is remaining steady or declining rather than showing improve 
rnent the government agency is under attack from Congress to justify 
me nay in which it has been spending funds the solicitors in the 
Community Chest Drive are not collecting as much money as they 
have previously a disruptive slate of tension exists among ethnic 
groups m the community John Doe feels he is not getting ahead 
m hie This type of sensitivity to a problem is frequently a reason 
nh> responsible leaders look outward for sources of help to get a 
deeper understanding of their problem situation and to find new 
principles and methods of functioning more effectively 

The image of potentiahty is another very important source of 
initiative Perhaps from their own imagination, or from observations 
of sitint.ons elsewhere certain individuals have an idea of how a 
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generalizing from, or staw that ore of 

lul clues It may sound somew p pj„jj,ng 5 concrete signifi 

the nays in nhich scientists about the findings 

cance for practitioners is to do adeq generalizations ivhich 

But this IS the case The spelling out o .Ues one o£ the most 

emerge from the stud) of a speafic situation p 31.5,5 of other 

helpfll means of relating these -'f ‘Vo^n masrpenuasion and 
situatiom The reports by Carttvng ( > ^ ^ examples 

Coch and French (9) on resistance “ = „f,l,e studies of 

of this Cartn-nghfs paper starts fro 

the selling of nar bonds during Wo W ,n%oUed 

French paper starts from a study of , , 1,0,1, cases hon 

in technological changeocer in “ 31,001 the phenomena 

c\er, the authors mo\ed to a level of th o practi 

the) hate smdted nhich makes it possi ^ analysis of their 
iioners to see hots the generalizations app ) organize 

problems This rs possible because P^^feen as reLant 

and interpret the data are concepts v\hich a ) 

and important in a wide vanetj of situations 

Studies of Widely Dislnbiited Phenomena and FopidaMns 

Studies such as those on the P'o to technological 

^tic and other kinds of leadership, on resis ^vork 

^nge, or on interpersonal relations between s Process which 

^ ha\e focused on aspects of behavior problems m m3n> 

3re important features of a wide vanet> o __.,ons based on re 

^ypes of soaal situations Theoretical genera— 

*^tch dealing with vsidespread phenom «:;ocial psvchological 

'■^nce for a widevanety of practical infrequent!) 

Indies which are focused on phenomena w i „eneralirations 

operating problems and situations are apt to y 6 studies 

“'fes.idJpLdapplicahi.uy Thtr d- noun^n ^.hm 

basic as contributions to the dev elopi « 

'o^uiion of some specific problem ivhich the scientists 

"rhere IS another closel) related wa> of research uiih 

Approach to his research helps faaUtate i e pr popub 

^^lon This 15 bv the selection for studv o si jndiistrial 

tions vhich * * ’ in societ) suci 


15 o) me bcjcunu** - 

are nidel) dnlnbutcd m socic ) 



584 The Application of Research Findings 

have been accepted and understood by peisons who it is hoped will 
utilize them and whether the research findings generalize to other 
problems and situations We shall examine some of these problems 
of applying research and then review a number of illustrative cases 
of social science utilization 


USING KNOWLEDGE AND THEORY DERIVED 
FROM RESEARCH ELSEWHERE 


As a person or organization turns to the scientific stockpile for 
help on a problem he is faced with a number of important questions 
about the applicability of research done in other settings To what 
extent and m what ways is his situation comparable to those m 
which the research was done? Does a theoretical principle hold for 
his situation also? Is the way of approaching the problem all he 
can learn from the previous work or is there also something con 
cerning the substantive content of the findings and generalizations? 

Unfortunately many persons do not leview these questions in 
looking for help from scientific resources hence they reject most 
scientific work and its implications because of certain manifest dif 
erences etween the situations or populations on which the research 
was conducted and their own Or they may uncritically accept all the 
findings and insights as relevant m their own situation and proceed 
unsuccessfully on this unrealistic assumption 

question which must be asked 
about what causes what prov.de 
lotilVn "”"'^‘"8 wtll happen tf ? 

underUZT *an the information needed to 

The sect.ons wh.ch fol 

low attempt to review some of the wivQ «F rVs i. i 

ttons of appl.cabihty of sc.em.fiV Z “'‘"'‘‘"f; “bom these ques 
1 , ^ ^ ^ scientihc resources and some of the ivays 

m which application can be facilitated ^ 


The Need for Sufficient Theory 

As the potent, al user reviews a p.ece of research done efsewhere 
he may or may no, d.scover that the setent.s, has done enouRh 
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interpret the research firtdirtgs and to observe the Procedu- 
used. Peer reassurance is illustrated here ) t le project in 

were able to talk about the feasibility and P^ental 

their own nontechnical language mt ^ eople we are." 
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^tscarch knowledge to bear on the anal>si5 o action. He 

Prediction of probable consequences of ., 5 „ring atul assess 

also be able to help set up procedures important, 

'he consequences of new lines of action. ! erbaps 
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work gioups, family units parent child relations, classrooms, etc 
It should be clear, however, that certain basic problems can be most 
effectively investigated under laboratory conditions with popula 
tions of volunteers who are willing to collaborate in scientific expen 
ments 


Channels of Communication 

But even if the research setting, type of problem, and treatment 
of the data are advantageous to the providing of important and 
relevant insights to a wide variety of planners and actors, a third 
type of problem may neiertheless be present There may be no 
actual communication of such relevant findings to potential con 
sumers who need them Effective communication must be established 
between relevant social scientific resources and the potential users 
of these resources One help m this direction is the work of the 
social science middleman,' the science writer A good social science 
interpreter is able to classify and synthesize research findings so that 
they are more clearly related to the problems posed by operating 
persons, and are related to a wide variety of problems so that many 
practitioners can find the material of relevance by reading such an 
overview and can learn where to find the sources of data Examples 
of this are the books by Watson (39), Murphy (35), and Marrow (33) 
Such overviews may also be presented as specially invited papers by 
scientists at the professional meetings of practitioners (25) Over 
views may frequently have the stimulus value of creating the image 
of potentiality referred to above, or they may provide someone who 
is sensitive to a pToblem with a direction in which to seek for help 
In the field of social psychological research, the implications 
of the findings for what to do in a specific social situation are often 
complex and likely to be tied up with fears and uncertainties about 
the consequences of the findings Therefore, more adequate and 
intensive methods of communication are required to stimulate an 
understanding and acceptance of research findings and their impli 
cations than m the physical saences For this reason, the processes 
of demonstration and of reassurance by peers are often of great im 
portance For example, a very important community project on the 
solving of mtergroup tensions sponsored an "open house workshop 
for leaders from other communities to come and help review and 
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diagnostically some of the differences between "last year" and "this 

year" as a money-raising situation. overcome 

It is possible that all these hurdles m'B"' “ve ^ 

had the consultant started out with t le ° ^ existed. Or he might 
for help, rather than assuming that ‘"1^ p. , modified 

have been successful in suggesting a po ^ 
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unite easy, however, for them to point out a differences, 
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such a scientist consultant usually finds that he needs to take the 
role of a trainer o£ his operating co workers in the scientific atti 
tude or scientific approach to thinking about the new operating 
problems which occur daily 


Illuslrations of Research UtthzaUon 

Now that we have reviewed briefly some of the problems and 
possibilities of motivation and communication involved in applying 
social science knowledge from other settings to specific operating 
problems let us look more closely at some examples of attempts to 
apply social science In a later section we shall summarize what seem 
to us some of the important general principles that underlie these 
illustrations 

The executive committee of a Community Chest asked a social 
psychologist to help them tram their fund solicitors After look 
ing into the past practices of the solicitation procedure, the con 
sultant recommended a number of modifications derived from 
research and iheor) For example he suggested that solicitors par 
ticipate more actively in the process of setting quota objectives so 
that they would feel psychologically more committed to their objec 
lives This was an application from se\eral research findings that 
persons who participate in a group decision are more likely to carry 
through the commitments of this decision than are persons who 
receive assignments or exhortation without participation Also the 
the plan included more careful consideration of matching the solici 
tor to his targets m terms of existing status and group relation 
ships Research on social influence has indicated the importance 
of prestige and reference group membership in exerting influence 
Another application of reference group theory was a recommenda 
non to develop a clearer rationale for expected size of contribution 
in terms of specific subpopulaiions to which each giver could sec 
himself as belonging 

The executive committee rejected all such ideas with a reaction 
that we made our quota last year by the previous methods so we d 
better use them again Psychologically they were unable to (1) 
accept the validity of the data from elsewhere as relevant (2) accept 
as realistic a goal of doing better than we have before (3) sec 



The Utilization of Social Science 591 

lui considerable consensus in 

problems During this phase, the practitioners ere pnma y 

ers although ‘’'"V ^oXaice the whole group 
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shattered They became eager to leant how to apply the discoveiics 
and innovations o£ the American plants in their own This iliustra 
non seems to be typical of many situations in which an individual 
or an organization has overtly expressed a desire to improve the 
present situation but is unable to accept various ideas for improve 
ment as relevant for themselves until they can see that the improve 
ments have been successfully executed by someone m their own 
league When this factor is accepted the data which have been 
iliscovered elsewhere are accepted as relevant and applicable to the 
analysis and evaluation of their own situation 

A RESEARCH APPLICATION CONFERENCE Let US tum nOW tO SCV 
eral illustrations of attempts to communicate research findings in a 
way that would clarify their relevance to social practice In the first 
example a group of adult education practitioners decided they 
would like to explore what help they could get in thinking about 
their professional operating problems from interaction with a 
selected group of social scientists They organized a two day confer 
ence which had the following design 

During the first phase of tlie conference the practitioners sat 
around the table taking a census of whai they regarded as critical 
professional problems attempting to arrive at some consensus con 
cerning the most important problems as they saw them They 
attempted to formulate the nature of these problems and the 
causes as they understood them During this phase the five or 
six social scientists representing various disciplines from anthro 
pology to psychology had the job of listening and keeping notes 
on their interpretations of the nature and the causes of the problems 
making cross references to any relevant social saence research and 
iheorizmg which might be helpful in shedding light on the various 
problems which were being explored 

During the second phase of the conference the social scientists 
took the center of the stage to interact with one another They 
shared and integrated their observations about the crucial dmien 
sions of the operating problems which had been discussed the way 
in which these problems might be related to deeper underlying prob 
lems on which research had been done and the kind of generilizv 
tions whidi might be tentatively advanced as guide lines for clarify 
mg tlie nature of the operating problems A w ide variety of researches 
and research-derived generalizations were brought to bear and there 
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about a problem They also felt that the ^fsome 
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which they were concerned At the end of the research review they 
would convene in committees of six or seven to share their thoughts 
and observations in order to formulate (1) questions of clarification 
to the social scientist, and (2) tenutive generalizations which they 
would draw from the research review concerning operating prob 
lems These they would test by getting the reaction of the research 
scientist to the validity of their generalizations and the possible 
extension of the generalizauons 

The research review which took about an hour and a half, 
integrated a great variety of studies under several research topic 
headings such as Leadership, Communication, Decision making, 
Participation, etc Under each heading a variety of empirical results 
were presented and a number of theoretical generalizations were 
formulated During the next hour, all the subcommittees held very 
active discussions, each committee having a recorder reporter The 
following period of interaction between the group reporters and 
the social scientists was also very active, although rather frustrating 
at times for the social scientist The subcommittees had been very 
imeniive and creative in applying and extending various research 
generalizations and he found himself m a position of challenging 
some of the applications which were being suggested He felt that 
they needed considerable testing before they could be applied He 
was active in suggesting boundaries to generalizations and dangers 
of oversimplifying the complexities of operating situations by think* 
ing only m terms of the interaction of two or three variables rather 
than of many additional ones which might be just as important and 
which had not been explored in the research which he had been 
reporting 

As a whole, the conference seemed to strike a balance between 
providing new insights into the analysis of operating problems and 
training in ' the scientific attitude" of thinking about some of the 
boundaries of generalizing from one situation and set of variables 
to mother The participanu m the discussion indicated a high 
degree of satisfaction with the results The research reviewer and the 
conference leader felt that there were a number of weaknesses which 
could be improved upon next time The) felt the whole problem 
of adequate research utihralion had not been grasped because no 
single illiistralion had been carried all the way through to an 
examination of the problems of acting on new ideas and insights 
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these trainees was to give them a specific operational “ 

solve in which their first job was to make decisions as to what kind 
of information they needed and from what type of social scientis s 
and to formulate a procedure for getting this information from sue i 
scientific personnel The scientists were then called in and the group 
had to make good use of them and integrate the material into a set 
of operating decisions and actions It was the judgment of the three 
leaders of this training program that the focus on the solution of 
specific operating problems was an indispensable requirement for 
the successful integration and use of the resources of the various 
social sciences One of the problems of such an approach was that 
of training the social scientists to enter into this group interview 
procedure in which they were asked to focus their thinking and 
background knowledge on certain functional questions lather than 
to present prepared lectures organizing the substantive content of 
their specialty around more general or abstract topic headings Most 
of them proved quite ready to make the adjustment and were able 
to contribute in a very fruitful way 

PRODUCING AN APPLIED SOCIAL SCIENCE PERIODICAL Another 

example of the interpretation of the social sciences is illustrated 
in the production of the monthly periodical Adult Leadership The 
professional association which has responsibility for the produc 
tion of this periodical has conceived its mam function as communi 
eating to lay leaders principles and techniques for performing 
their job of community and organizational leadership It was felt 
that this objective called for drawing on the resources of the social 
sciences in order to produce valid content for such a periodical On 
the other hand it was recognized that most social scientists would 
not have the time or the skill to communicate clearly through the 
type of writing demanded for such a popular periodical Therefore 
an Operating Committee was established which was composed of 
representatives from the social sciences from training methodology, 
from magazine publication and art layout from public relations and 
from the operating field of leadership practice This group was given 
the major responsibility for thrashing out with the small periodical 
staff the major policies and procedures for the content and produc 
tion of the periodical 

In the early stages of production fervor, a number of things 
happened which helped to put the microscope on certain problems 
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the facts to be voted upon and to get out the vote In planning 
the campaign the head of the local PTA sought the advice of a 
social psychologist on what the PTA should do m order to be cer 
tain that the voters acted when the time came to vote 

This social psychologist had had research experience during 
the war on the problem of why people did or did not buy war bonds 
He knew that the studies of the several war bond campaigns had 
shown that the mass media were useful in helping people to under 
stand the reasons why it was important for them to buy bonds But 
he also knew that a major finding of the studies of the campaigns 
was that people were likely to act by buying a bond only when they 
were personally asked to do so 

In advising the local PTA he drew upon his knowledge of these 
and related results He urged them to plan a campaign utilizing both 
the mass media and personal solicitation but with emphasis on the 
latter He suggested that they ask the local newspaper and radio sta 
tions to carry as much information as possible on the facts of the 
situation and to repeat this information in somewhat different 
stones on different days so as to increase the probability that people 
would become aware of the election and the issues involved But he 
emphasized in talking with the P T A that if they wished to moti 
\ate people to participate in voting u would be very important to 
hive each household called upon by a volunteer from the P T A to 
encourage every eligible voter in the household to vote Acting in a 
vigorous manner upon his advice the PTA organized an informa 
tion program about the need for additional schools and the cost of 
these additions which was widely disseminated through the coopera 
lion of the local mass media Most important of all however the local 
PTA organized a campaign in winch every household through 
out the city was called upon In these personal calls on individual 
households neighbors of the person called upon gave him facts 
about the situation and urged him to be sure to vote in the forth 
coming election The effect of this rampaign with its house to house 
solicitation was a large vote in the election and one which was 
osenvhelmmgly in favor of the additional schools and of the bond 
issue to finance them 

AN IN SERVICE SEMINAR A final illustration of methods of re 
search interpretation is a combination of scientific discoveries from 
other situations and the findings of research conducted within an 
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interpret relevant research results and theoretical generalizations 
developed elsewhere The conscientious research scientist is quite 
likely to take the point of view that this is an impossible task with 
out an additional comprehensive research diagnosis based on meas 
urement of the present situation Frequently this is correct and 
this will be the focus of the second section of the chapter Fre 
quently however this approach is not feasible and would require 
quite an impractical delay in the decision making and planning 
demands Therefore the social scientist consultant is frequently 
faced with the necessity of getting the picture as quickly and as 
objectively as possible The following case illustrates two techniques 
of tackling this problem 

The social scientist was asked to consult with the administration 


of a large hospital on any relevant knowledge there might be which 
svould be helpful in reduang the iniergroup tensions among the 
nurses doctors and attendants m such a way that there would be 
unproved care of the patients During one afternoon the consultant 
conducted three problem census group sessions In one session 
he met with a group of medical personnel in another with a group 
of nursing personnel and in the third with a group of attendants 
In each session he shared his problem of wanting to get valid data 
on the problems each subpopulation fell it had of cooperation 
with the other subpopulations m the interests of serving the patient 
e selected three persons at random m each group explaining that 
while he intervieued them in front of the rest of the group he 
would like to have all the other members jot down whether they 
a^ee wit or disagreed with the responses of the interviewee and 
what modifications and additions they ivould make in their own 
response The data ^\ouId be reported orally if there was time but 
would be handed in if there was not sufficient time 

The interviews were then conducted with a preplanned set of 
questions and proceeded slowly enough so that the rest of the group 
ormu ate the variations m their responses Most of them were 
ve^ active m tlo.ng th.s The mterv.e.vs and supplementary data 
tnd.cated that most ct the tensions and low moralL stemmed from 
.nteraction s.tuat.ons wh.ch occurred tn the daily rout.nes o( hos 
p.tal hfe It seemed to the consultant from these Reports that prob 
ably a good deal of the d.lEcnIty had to do w.th noncommumcat.on 
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o( expectations, -nparttc.pat.on .n deacon making, or noncom 

munication of decisions which a rcle\ant phe 

To check further whether these «^^^^“\rotn each role 

nomena, the consultant recruited set e . „s,n2 role playing 

to reaeate some typical ^ situations the consultant 

techniques From his observations o variables which were 

lelt fairly secure about some o£ t e ma] relevant research 

involved and was able to summarize make some general 

Irom other social hierarchical situations, “ suggested 

interpretations about the problems actively involved 

liom experience elsewhere that key I«rs participate in a 

m an actual research project m whic t le problem 

more thorough diagnosis of the dynamics ^[,,5 chapter 

situation This step takes us into the nex -crating situation 
on the utthzation of research done wit 1 ii„.trates is ^hat the 
The mam point which this brief jeast crude data 

tonsultant can use a variety ot techniques appropri 

on the particular operating problem so t ^h and theorizing 

z'dy mobilize the scientific resources o , „ insights about 

lone elsewhere which may be relevant in st.mulati g 
’lie present problem 


'"lerpremiive Review 0/ Case Illuslrations One of 

important variations in the riety of illuslrations 

">e important facts we can note from th z partici 

*toie IS that the psychological readiness * diflcts greatly from 
pants to use social science and social scie -ome cases the 
ntuation to situation We have noted tha ^ Pnowledge and 


“'u to use social science ana ssa.-— cases u‘v “r 

“‘“ation to situation We have noted tha j. now ledge and 

“'"g personnel have taken initiative to seex ,0 poten 

"■ail o! applying u, whereas in other cases mforma 

‘'alitics does not exist Also where this des t curiosity 

“on does exist, m some situations it is a gen » ^^j^arch application 
•'’out possibilities of application (such as 1 cither than a sear 
“e'ercncc and the research review contcre ) campaig 

'“'''clp on a specific problem (such M „w The ’ ’ 

'•'le improvement of the morale m the . ^„ctitioners ask hn . 
^al scientist is somewhat different when P 
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“Tell us in general what’s going on and what new things are being 
discovered in your field," from what it is when he is asked, Do 
you know of any knowledge that will help us in solving this prob- 
lem?” We can note another difference in our case illustrations 
between those situations in which the social scientist is primarily 
an information-giver and situations in which he is an active con- 
sultant in guiding the process of research application and doing an 
active job of interpreting scientific method and the generalization 
of scientific findings from one situation to another. 

Our case illustrations emphasize the importance of this more 
active role for the scientist in applying scientific knowledge and 
theory in an appropriate and effective manner. This does not mean 
that many important applications have not also emerged from a 
practitioner’s reading some book of social-science interpretation or 
hearing some research-interpretation speech. Such illustrations are 
not very likely to come to our attention. It is our hunch, however, 
that the complexities of social-science research utilization are so 
great that in most cases it requires the very active teamwork of 
motivated practitioners and well-oriented social scientists to bring 
about intelligent application of available scientific knowledge and 
theory. 

THE PROCESS OF RESEARCH UTiLizATJON, A revicw of the fore- 
going case illustrations from a different point of view reveals a 
number of things that rather typically need to happen it successful 
aphcatton of scientific research and theory is to result. Even 
though the situations did differ greatly from one another, the fob 
lowing common elements seem to be necessary or at least highly 


(1) There needed to be the motivation to seek and use scientific 
resources. In situations where this motivation did not exist, it needed 
I- •’>' demonstrations of potentiality, by complacency 

ock, and other approaches. This seemed to be the first step before 
further progress could be made. ^ 

nn/srlt' active process of redefining and reformulating 

“ '•>« 'he relevance of scientific 
research done elsewhere could be perceived. In the research-appli- 
^t on conlerence the team of social scientists did an active jo/ of 
first listening to the practttioncis- sutements of their problems and 
then attempttne to break these problems down and refomtulate them 
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t TPiearch-review conference 

for scientific analysis. In the case o practitioners took 

and of the administrative operating problems as 

the initiative in attempting to reia e 
they saw them, to various research pro 

on. *:«nc vie saw that the social 

(3) In nearly all o£ the 'to do an active job o£ 

scientist, in order to try to be “ P ’ practitioner saw it 

getting oriented to the action pro em ^electing appropriate 

h order that he might do an “'tticuUr fituation. He 

scientific resources for application to t i 

had to be a skilled listener and interro^ interactions 

the persons involved recreate some o t e orientation, 

of the problem situation in order to get an more thorough 

In many cases of social-science ug {gets about the oper- 

diagnostic research job is necessary to g^j jj, [he next section 

«ing situation. This type o£ ’* ' .jlization of research con- 

of this chapter, where we deal with t 

iucted within the setting. . .rjentisl to communicate 

(4) It usually is helpful for the social s dynamics' 

a general orientation or "way of loo '"’S data. This cer- 

>s a framework within which to interpr in the behav- 

^inly suggests that some types of effective applitutio" o 

ioral sciences is needed as a backgroun 

r^carch knowledge to specific problems. ^blems in terms of 

(5) In addition to reformulating active job of thinking 

"levant scientific findings or variables, a knowledge and 

t!»ut generalization and applicability o scientist, as a con- 

theory is required. This requires that die mctbodolog)’ 

"Itani, help the applier gain an understa comparability o 

“I tesearch application by facing such que ...pamics, extrapolation 

P'^pulations, comparability of , >it„ations an, 


* -foirtiiuns, comparability of siiuaiuj"*'* ^ and experi 

theoretical generalizations to differen 

;„,r TIPW solutions. 


■“‘i-urcucai generauzuiiwin* - 

*’ttnial-mindednes5 in trydng new .i, application bas been 

(fi) One other important aspect o . j-dated to a gi'cn 

*^«ntioned. The scientific knowledge t'diic l or 

. ui^m may be oi 


knowledge winch is ,.-„r 

-ing situation or a problem . a^c tbe way tfit^> 
?""ple. it mav r.!vc insiebt into "wliy tlo'jP ,;„,ntion could !« 


^Petaiin 


I •c-. .k may give insight into * why jjc siiuaiion could l>c 

give pc4ective on "ways in w ind, d c ^ ,houi 

‘’‘’’'""t." or it may give information about 
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changing the situation In all cases, successful application requires 
that the applier interpret, plan, and execute specific steps of action 
in his own situation This requires creative and realistic thinking 
about ‘what would happen if Helping formulate correct hypoth 
eses and plans at this point is a challenging task Even though the 
directions for application may be well understood, it may be neces 
sary to acquire new technical or human relations skills to execute 
the desired steps 


RESEARCH METHODS APPLIED TO PROBLEMS 
OF ORGANIZATIONS 

Social science research is being done increasingly in real life 
settings and on problems of major social significance This trend is 
likely to continue as the methodologies and substantive findings of 
the social sciences become more extensive and more able to shed 
important light on complex problems But if an organization is to 
derive full benefit from any research done on its problems expen 
ence suggests that the work must be organized, planned, and con 
ducted in accordance with certain patterns and principles The rest 
of this chapter is devoted to a consideration of these patterns and 
principles It is written primarily from the standpoint of a research 
staff which conducts sample surveys m, or for, an operating organ 
ization but the principles stated are believed to be Renerall} 
applicable ^ 


Establishing a Working Relationship 


CRATING A COOPERATIVE ATMOSPHERE An important problem 
in conducting research m an organization concerns the attitude of 

and Its personnel Since 
organiza ,ons vary greatly fte.r trad.tions and values a researcher 
may wish at times, to conduct research an organization whose 
values and conv.cl.on, differ m tmportant ways from his own In 
such circumstances, tt is important that he act in a way that makes 
clear that he respecu the values of the organizat.on and of its 
members even though he may not completely agree wtth them In 
planning and conducting the research and m reporting the results 
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to the organtzatton, he will obtain organization 

pla^s a sensiiive appreciation of the h-niself need feel com 

Vhts does not mean that the researdier himsel^^^^ 

pelled to accept and be guided perso , j,,s personal 

organization He can maintain his own v direct conflict 

integrity provided, of course. ‘ ^iisearcher s values and 

lulh those of the organization When usually is 

those of the organization are in direct conflict, he usua y 
not conducting research for the organization -j,„^point for con 
The major purpose from an improvement al 

ducting research is to improve us operaiio Qjganizations are 
''■ajs requires change This problem is not ^^ggarch does is to 
continuously undergoing some change ^ problems 

inaease the amount and magnitude o c consequently. 

'nvoUed m changing the activities of research Bndmgs 

present when attempts are made to app " , j^gg are seldom 
It IS common experience that orders y pj^anon and its 

luificiem to produce effective change m ^ ^vhich make some 
functioning Other procedures, including i gj-gons who need 

use of participation, are usually require others are those 

seek the participation or cooperation oi ^,ght bring 

persons who possess information as ^ research u u th® 

•niproNeraeni When this information is base 
researchers consequently, who are ^__ation of the others 

fern of obtaining the participation an gggfiilly Moreo\er, 

d the research results are to '’'=?PP”fll, d.eof die neuidca 
applying new ideas requires not only a >-n research 

ut also a full understanding of the °!ninEr participation not 

'htt consequemly, faces the problem of o desirable changes 

““'y to facilitate cooperation in bringing “ , ,he best avail 

™t also to be sure that the changes sought r p current research 
“'kthinhing based both upon past experience 

andings d to a consider 

"fch of the rest of this chapter vmU Cooperation m 

’“on of b-... 1 .X.. i.nH of participati .,,„iriDatioii 


. _ . ■ will be Cooperation m 

-- how to obtain this kind of P“''“"P’’V,°"c 5 t participnt'e'’ 
and achieving change grows out ideas that 

“•t lull recognition and appreciation of . Cooperation i 

of P“P’o ‘■"‘■'.'ir" :nmtng 


mu recognition and appreciaiiu* „,,,buie 
»’“y kinds of people involved can “Utr-bu 

Seated by manipulation— at least not 
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A\oiDnG RESISTANCE an organization is contemplating 

having research undertaken for it. there ma> be some important 
persons m influential posiuonstvho view the proposed research with 
some reservations It is well not to proceed with the research until 
these cntiasms and reservations have been evamined full) and m 
relation to the advantages and disadvantages of undertaking the 
research Their resistance will manifest itself sooner or later, and u 
usuall) IS better to have it out in the open and full) faced eaiTy m 
the proceedings Often, if this is done candidlj and unemouonaU), 
these penons will become more and more involved in the research 
and increasing!) favorable m their attitude toward it If resistance is 
Ignored or brushed aside u is Iikel) to result in efforts to stop the 
the research if difficulues are encountered, or it ma) result in at 
tempts to block applicauons of the research 

cRE-VTiNc REAUsnc EXPECTATIONS Just as somc persons are 
undul) skeptical as to the probable value of the results that research 
will )'ield, others are unduly optimistic. This latter group tends to 
have unrealistic expeciaiiom as to what research can do for them 
or for the orgamzauon This can result in serious difficulties for 
there are aspirations that even the best research cannot possibl) 
achieve If people within an organization maintain these unrealistic 
expectations, the) are bound to be disappointed with the results 
obtained from an) stud), no matter how good these results *ire 
Such disappointment ma) lead them to reject the idea of using 
research in the future 

In order to avoid the disappointment which occurs when unreal 
expectations exist, it is important to create expectations that are 
modest in relation to the probable contnbuuons from research The 
creauon of these moderate expectations is best done during the 
planning stages of the research project This can be done while 
the problem to be studied and the probable character of the results 
to be obtained are discussed When expeciaiions are modest, the 
value of the research results are hkel) to be greater than antiapated 
This encourages live further use of research 


Organization of Research Relattomhips 

AN INTERN \L STAET VS AN OUTSIDE RESEARCH ORGANIZATION 
There is neither a simple nor a uxuversal answer to the quesuon of 
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whether research depends upo/the prob 

r»’.s s™. 

ever, which may be useful in he ping observations by 

specific situation It may be well to 

stating some questions and commenting up^^ organizatidn or is a 

Is a research staff located ^nrp likely to 

“ "O'* 

'■r.ifri. .--n'-r r»,: 

a well known research institutio , 
greater success any pressures likely to result 

objectivity fundamental dimensions 

(2) be able to focus the research on the t 
of the problems being considered? 

Often the officers of an orgamza immedi 

problem involved that they ten -.otoms and basic 
L problems and often “"'“^^^^kely m b^ 
causal factors The research is {.(-tors rather than 

tive If It IS focused on ^--^^““^".uJarly if n 
symptoms An outside res ^ research institu 

comes from a well known and p research 

tion, IS often in a better position immediate 

on the causal factors rather than sy p 
operating problems of a transitory ^ times be 

search staff of high prestige an po research on 

equally successful in being “ ^ mude research 

the basic problems But it is d assumptions and 

staff to question and do ° management is 

underlying philosophy upon which its top 

operating , , _ of all its rami 

(3) have a better knowledge of the pro 

fications? rr jjiay at times be handi 

Although an inside research ® nroblem m terms of 

capped by not being able to tac e ^g^gy^j^eless usually 
us more important * rch staff about the 

knows more than an outside res research start 

problems of an organization 
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usually has an appreciable amount of information avail 
able about the problems of the organization, their history, 
and some of the major developments with regard to them 

(4) receive the full cooperation of all of the persons involved? 
An inside organization usually has closer contacts witli 
the personnel whose cooperation will be important to the 
research If an atmosphere of confidence and trust perme 
ates the organization an inside research staff is likely to 
obtain the better cooperation On the other hand, if 
conflict and a substantial amount of fear and distrust are 
present, an outside organization with a reputation for 
objectivity and integrity is likely to be able to obtain the 
better cooperation In general, any research operation 
which depends upon obtaining data from people s re 
sponses will obtain full, accurate data only when the 
people whose cooperation is required feel that they can 
trust the research staff 

(5) have or be able to obtain the personnel required to do the 
research, including personnel with the specialized compe 
tence that may be required? 

The desirability of creating within an organization a 
trained research staff including specialized technical per 
sonnel will depend in pan upon whether this is a one 
time study or one of a continuous senes of related studies 
If the organization is continuously conducting a sub 
stantial amount of such research, it is often advantageous 
for It to create its own research staff If it is a one time 
study, or if the volume of research undertaken is limited 
the organization often obtains better and more econom 
ical research by using an outside research staff rather than 
creating one for the single study 

(6) be able to exercise the influence required to have the results 
of the research effectively used? 

In some situations an inside research staff has the power 
and prestige to assure that the results are likely to be 
used In other instances an outside research staff with an 
excellent reputation is more likely to be able to exert the 
influence required In deading \shether to use one or 
the other or both in a cooperate e arrangement consid 
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™ b, gi.™ bo .b. p-fh." 

and the amount each is j g important as 

sion of the problem IS proba > _„parch staff. This 

the question of an ^ erson to whom the 

question concerns the status ot p 

research staff reports. If he ts a top J 

to he sufficient influence present - 

consideration of the results. m,,rh less likely to 

importance, the research findings are much V 

receive consideration. ^ procedure which has 

SELF-SURVEYS. The self-survey (38. ) P . widespread 

merit where the problem is not too comp ex organization 

participation is advantageous. With th- P^ and as- 

iiself conducts the study with appropr step-by step 

sistance. Usually a manual is available oreanizations also 

basis, the self-survey is to be conducted Most^orpmra^^^ ^ 
obtain the assistance of a competent so consultant answers 

consultant during the period of the survey. survey. His 

questions and provides advice on how to conduc^^the^s y^ 
services include such tasks as helping to trai P inter- 

Usks required, advising on analysis participation, the 

pretation of the results. In addition to fa ^ study 

self-survey usually has the advantage of cos i ^ outlay. It 

done by a research staff, or at least of shortage of 

aUn ,.,.r>H.irtinR a study wnen ui 


ae by a research staff, or at least of mvo vi g shortage of 

^ho provides a means of condurting a ^ ' , other kind 

^ined researchers would make it impossi e 

-U nnlv on a limited variety 

The self-survey, however, can be us y possible to 

u' problems. They must be problems “c '' will, never- 

e\elop a reasonably simple self-survey pro j-ontinuous danger 

*«^ess. yield results of satisfactory actmracy. persons will 

‘n using self-surveys is that the utilization of 
yield results containing serious pitots. In ^ 

P^cially likely to occur or in which 'Inwise to use the self- 

’^ould be particularly disastrous, it woul c 

It has been ob- 

hierarchical level of the research j. supervision of 
that when a research staff is under ^ ^ research, the 

person whose operation is being affected y 
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r;.tr. :is 

Lai sciences, and for the same reason As the 

lead to improvements the head oE the operation most afTected m y 

feel threatLed Insofar as he feels threatened, he is 

have the research discontinued or the results ignored When tms 

occurs he may discontinue it or bury the research findings it ne 

has the authority to do so Consequently, the research staff m an 

organization should neither report to nor be directly responsi e 

to the person ivhose operation is directly alTected by the researc u 

should report to the next higher echelon 

There is another important adtantage m having the researc 
staff report to the next higher echelon When the research sta 
reports to the man tvhose operation is affected by it, he may hmd 
the scope of the problems studied or he may order it to produce 
findings to prove him right m what he is doing Such restrictions 
make it impossible to conduct research effectively 

Although the research staff should report to a sufficiently high 
echelon m an organization to protect its stability and integrity* 
all of Its activities need not be conducted through channels via this 
level Usually it is well to use the formal channels in agreeing on 
the nature and scope of a research project, but after that is done 
there is a distinct advantage m the research staff s establishing direct 
and close contact and communication with all the echelons and 
groups involved in the research Even in using the formal channels 
in agreeing on the nature of the research project, authority should 
not be used to force the research on the units involved It usually is 
unwise to undertake research for or in a unit which does not gen 
umely want it When people are forced to cooperate, they are likely 
to give distorted information 


Problems in Research Design 

BASIC RATHER THAN SUPERnctAR VARIABLES It IS HOt Rt all COW 

mon for an organization to request that research be done on some 
problem about which it curtenay « very much exercised and which 
It feels important Often, however when this problem is examined 
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it proves not to be the ''deriving problem. It may be 

may be a symptom rather than , , Or it may not be stated 
only one part of a fundamental research, 

in dimensions which permit . ' jj^^nose the situation and 

The first task of a research staff is ' research. Discussing 

to prepare a clear statement of o jec i organization 

the problem fully with the officers a problem in ditnen- 

facilitates this diagnosis. Attempting ‘ j j eonceptualizations 
sions based upon the best available theoret 

also helps. j of the objectives of the 

The final statement of the . ,tion. Often the discus- 

research must be acceptable to the org ^ recognition and 

sions involved in diagnosing the prci e objectives. Some- 

acceptance of the problem as stated m discussions do not 

times, however, this does not occur. , d, the research staff 

lead to an acceptance of the problem devoted to a 

may have to start on a P'*°‘ “ !hn oreanization recognizes an 
peripheral problem but one f results obtained m this 

about which it is much concerned. Fro demonstrate the nature 

pilot study, the research staff usual y research should be 

of the basic problem upon which t „ins a clearer under- 

omcentrated. Often the research stall ■ ^^^dy. 

standing of the basic problem as a resu ;^PPUED objectives. 

aEiATioNSHip BETWEEN THEORETicA research devoted to 

It is impossible to emphasize sufficient y done, 

he operating problems of an agency ” problems or wit 

ts'ill not be concerned with the symp finding sp' 

... a__ solely n 


the operating problems oE an agency r problems or wb 

not be concerned with the symp finding speci c 

tainutiae. Nor will it be cumulating which points 

^swers to specific problems. with a specific oper- 

to the advantage of designing rese^c generalized an 

^ting problem in such a ^vay that the res 

applied to other related situations. ,„pncv seeks only to fin 

IE the scientist doing research Eor a . run into serious 

specific answers to specific problems, te is there are so many 

difficulties. One difficulty, for example,^ '' 

*P«ific problems that he will be hopete y of the 

■hat the cost of the research is hkely that by t 

'Pacific answers. But particularly ,ns„er to a particular 

time the research has provided a speci 
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problem the situation ivill have so changed that the original prob 
lem IS no longer the problem New ones have replaced it (11) By 
inference this indicates that research designed to meet the long 
range problems of an organization will be more valuable and have 
greater application than fire fighting research designed to meet 
immediate problems 

The great value of generalized knowledge for dealing with 
specific problems was well stated in a remark attributed to Kurt 
Lewin Nothing is so practical as a good theory In the design of 
research focused on major variables the probability of significant 
findings IS increased if the best available theory is used as a guide 
as to what to measure and what relationships to test The better the 
theory used in guiding the research design the greater is the prob 
ability of finding marked and important relationships Obviously 
the more that the research discovers about those major variables 
which have a marked relationship to the problem being studied 
the greater is the contribution of the research to solving this and 
related problems 

Generalizations or statements of principles which summarize 
the marked important relationships discovered in the research have 
two valuable uses They serve as guides to help solve problems like 
the one upon which the research was focused They also make a 
contribution to available scientific knowledge and the development 
o t eory Cartwrights Some Principles of Mass Persuasion is a 
good illustration (6 7) 


In situations in which important changes m the character of the 
problem are likely to occur between the beginnings of the research 
and the availability of the results u is necessary for the researcher 
to take this into account in designing his research In addition to 
oncemrating research on the major variables involved he often 

rdenn" 1 *31 the results will 

adequately cover any reasonable range of change that may occur 
m the situation during the time required for the research One way 

slsfe ? r 'a “ »tH yield results 

satisfactory for dealing analytically with two or more widely differing 
ituations If these assumptions involve situations more extreme 
than any that is tke y to occur then the actual situation at the end 
of the study will fa 1 between the extreme situations assumed By 
bracketing his problem in this manner and having adequate data 
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to deal with a range o£ the situation 

maie valid and useful derivations from his findings 

that exists when the research findings ecom national re 

arsEAKCH ON PiuNCtPixs AND '".“^earch staff to 

search (see Case A, pp 620 630), it is we research is to 

emphasize and reemphasize that the methods and princi 

discover the relative effectiveness of nerform a policing 

pies and that the study is m no way an . principles and 

taction The emphasis must be on discovering P” J „h.ch 
methods work best and why. and not on finding and repo g 
individuals are doing their jobs well or poor y 
TT^T„_. ^ve. madc clear ti 


viduals are doing their jobs well or poo y rigorously 

Unless these objectives are made ctxc obtain 

adhered to, it will not be possible for t e re organiza 

the full c ' " 


d to, It will not be , pie m the organiza 

-^-1 cooperation that it needs fro s,a(j to make 

turn being studied It is important to „.,,„nnaires, and other 
dear to every person that the interviews qu confidential 

data obtained from each person will be p collected for 

People need to know that these materials mil 

purposes of statistical analysis and °ndividual 

"hich specific answers were given by w ic i -jjgriy given and 

The commitment of confidentiality mus ^^If not 

sitictly adhered to This at times may requi m 

“> report separately the data for very ; dividual members 
^ight reveal the attitude and answers oi 

the small group principles and 

An orientation focused on discover! g pgrsons who nia> 
^ethods of organization and leadership research is to 

threatened by the research If they ee successfully, they 

‘^arn how to help them to do their job m increases 

visually are eager to cooperate This 05 e rather than to 

they see the research results used for t us P successful 

'^charge or demote those whose work at pr 


d«u 


‘*»unng Use of Research Results ^rnrUDES 

inducing cooperative father 1 co\ernniental op- 

^urements of any commercial, industria ’. gjonc well and 
almost always show that some things a research 

things arc not being done so well In examining 
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results, the officers of an organuation can take primarily either a 
constructive or a defensive attitude toward the data Fortunately, 
most officers take a constructive point of view Occasional!), how 
ever, a company officer or supervisor takes a defensive attitude and 
immediately becomes fearful when data are obtained which show 
that his operation is not now functioning in the best possible 
manner His impulse, as soon as he has seen such research results, 
is to lock them up immediately so that no one else can discover 
that the operation is functioning imperfectly Most company officers 
or supervisors take the opposite point of view when looking at 
similar data Their reaction is to look at the results which present 
a favorable picture with pleasure but to look at them hastily They 
then turn with genuine enthusiasm to the results which indicate 
where and in what manner the operation can be improved They 
immediately share this information wiili their colleagues, subor 
dinates, and all other relevant penonnel in order that the necessary 
steps can be taken which will lead to further improvement in the 
company s performance 

As we shall see, there is much that both the officers of an organ 
ization and the research staff can do to prepare and assist the per 
sonnel of the organization to take a constructive rather than a 
defensive attitude toward research results 

PARTICIPATION IN PLANNING AND IN INTERPRETATION If people 

are unfamiliar with a research project and know little about it, they 
are not likely to understand the findings or be interested in apply 
ing them Personal involvement not only decreases the barriers 
to the use of data, it increases the probability that the results will 
be understood and accepted Particularly important, hovsever, it 
yields positive motivation to apply the resulu This involvement 
should include all those who can influence the application of the 
results and should begin at the very outset of the project and 
increase as the project reaches the analysis stages To wait until 
research results are available before attempting to obtain participa 
lion represents a failure reall) to use participation and is likely to 
lead to the full or partial rejecuon of the results 

The effectiveness of participation and involvement depends 
upon the rate or timing of the efforts devoted to this purpose 

ere seems to be no substitute for taking adequate time at many 
points in the process The first point occurs when an organization 
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implicit demand that the others make an immediate redefinition of 
the situation, serves only to increase the emotional resistance and 
the amount of time ultimately required to get the findings accepted 
and used It is best to give the individual or group ways to save face- 
let them explore all of the different possible meanings which the find 
mgs might be assumed to have— before going ahead One of the more 
important things that an outsider can effectively do here is to pro 
vide the individual with the motivation to re examine his psycho 
logical field to see if there are not even better interpretations to the 
perceptual clues he has been getting and piecing together than the 
pattern which heretofore has satisfied him (32a) 

(7) The results should be in simple, nontechnical language 
Shop language and graphical presentation should be used as much 
as possible This facilitates self analysis by making the group realize 
that the data deal with their situation and are not something be 
longing to the research organization 

ANALYZING DATA IN CROUPS The procedure described in Case A 
(pp 620 630) involves working with groups rather than with indi 
viduals Lewin (19 20) and his students (10) have emphasized the 
power of the interacting forces exerted by group members on one 
another Mann (31) finds that this is particularly applicable to a 
groups discussion of data dealing with its own work situation 
Participation in group discussions and group decisions concerning 
future action sets into motion pressures for action which are more 
effective than when individuals alone are concerned By working 
with and through groups, use is made of these continuing group 


important for several reasons 
( ) rough group discussions the findings can be examined 
perspective because the group brings to the data expen 

vidual “ =‘"y 


f /"“P discussicm, by allowing the pooling and exchange of 
loon “'’O provide the psychological situ 
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w^vs of "crv und improved 
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(S) The discussion by groups of the research data compels all 
members of the group to recogniie openly the existence of the 
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accept or reject the data Also, as further data are reported to them 
and progressively build a clearer and clearer picture of the results, 
the officers follow it with interest During this period, they can test 
the validity of the results by using other evidence or clues This 
testing and discovering that the results are valid facilitates their 
acceptance 

People find it difficult to make a major change in their thinking 
rapidly It seems to require time for each of us to test new ideas and 
new results and gradually to discover their validity and to accept 
them Not until then are we willing to build decisions upon these 
new findings There seems to be no substitute for time in this proc 
ess Whenever pressure is exerted to achieve changes in points of 
view or m thinking in unduly short periods of time, there is likely 
to be strong emotional resistance to it 

SELF ANALYSIS TECHNIQUES AND THE USE OF RESULTS ^ PartlCipa 

tion in the form of self analysts is more likely to be followed by 
changes than if the analysis is made by someone else This point 
and the specific procedures used to implement it are not new Clin 
ical psychologists are well aware of the importance of self analysis 
for bringing about change (36) Some of the procedures which ap 
pear to be particularly effective follow 

(1) One important dimension of self analysis as used, for exam 
pie in Case A (pp 620 630) is that it starts from and centers around 
^jjective measurements This tends to keep the discussion in an 

icctive atmosphere There are no statements, reports, or recom 
ndations by outside experts to which an individual or a group 
a i take exception there are only unemotional, objective data 

(2) In the senes of meetings described in Case A, no expert tells 
^ group what the data mean or what its problems are The inter 

j are worked out by the members of the group themselves 
the representative from the research organization there to 
answer technical questions which the group leader cannot answer 
Prior to the meeting members of the research staff always study 
the data that are to be discussed At times in the meetings the 
research staff members may ask questions about the data to help 
focus attention on what appear to them to be important points that 


iThe following three sections have been taken from Floyd C Mann and 
Rensis Uktit The Need for Research on Communicating Research Results 
Human Organization 1952 11,4 15 19 ° 
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implicit demand that the others make an immediate redefinition of 
the situation, serves only to increase the emotional resistance and 
the amount of time ultimately required to get the findings accepted 
and used It is best to give the individual or group ways to save face- 
let them explore all of the different possible meanings which the find 
mgs might be assumed to have— before going ahead One of the more 
important things that an outsider can effectively do here is to pro 
vide the individual with the motivation to reexamine his psycho 
logical field to see if there are not even better interpretations to the 
perceptual clues he has been getting and piecing together than the 
pattern which heretofore has satisfied him (32fl) 

(7) The results should be m simple, nontechnical language 
Shop language and graphical presentation should be used as much 
as possible This facilitates sett analysis by making the group realize 
that the data deal with their situation and are not something be- 
longing to the research organization 

ANALYZING DATA IN CROUPS The procedure described in Case A 
(pp 620 630) involves working with groups rather than with indi 
viduals Lewin (19, 20) and his students (10) have emphasized the 
power of the interacting forces exerted by group members on one 
another Mann (31) finds that this is particularly applicable to a 
groups discussion of data dealing with its own work situation 
Participation m group discussions and group decisions concerning 
future action sets into motion pressures for action which are more 
effective than when individuals alone are concerned By working 
with and through groups, use is made of these continuing group 


The^oup situation seems to be important for several reasons 
u -1 S^otip discussions the findings can be examined 

in a broader perspective because the group brings to the data experi 
enre t „ 


ence that is richer and 
vidual 


(2) Group discussions, by allowing the pooling and exchange of 
his wider range of information, also provide the psychological situ 
ation m which superiors and subordtnates at all levels can discuss 
possible solutions and thus gtve one another new and improved 
ways of viewing and solving their problems 

(3) The discussion, by groups, of the research data compels all 
members of the group to recognize openly the existence of the 
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the lag between the development of new knowledge and its actual 
use 

USING DATA SO THAT IT PRESSES FOR ACTION BusmeSS exeCUtlVCS 
and government officials who have made extensive use of social 
science research often point to the fact that they use the research 
results m such a way as to make the data do the work of pressing 
for action ’ 

The Director of the War Finance Division during the last 
war used research results \ery effectively to press for action One 
illustration will serve to show how he did this He knew, for exam 
pie, from his experience as Chairman of the X State War Bond 
Committee, that personal sohatation was essential if substantial sales 
of bonds were to be made to large numbers of individuals He discov 
ered, also after he became Director of the ^Var Finance Division, that 


the chairmen of many other states did not accept his experience as a 
guide to what they ought to do They shuddered at the thought 
of having to reauit and tram tens of thousands of volunteers to 
serve as solicitors in war bond campaigns Consequently, when he 
urged them to use solicitation and cited his oivn experience, they 
would solemnly assure him that their state was different from 
■” state and that personal solicitation was not necessary to sell 
bonds in their state The result was an impasse and, at first, seseral 
states did not use personal solicitation 

A study of the effectiveness of the Second War Bond Drive 


provided the director with data such as that shown in Figure I He 
ad a brief pamphlet prepared which showed these and related 
results He distributed these pamphlets to all the state and county 
war ond committees He also had these results presented in the 
regional meetings in which the plans tor the Third War Bond Drive 
were discussed and developed The net effect of using the data m 
^ bond committees convinced themselves of the 

ue o so icitation They recruited and trained a much larger 
poup of solicitors As a result, the number of people who were 

th a n ^ bonds increased from 25 percent in 

the Second Dnve to 50 percent m the Third, and the sales of Senes 
it Bonds almost doubled 


The impartiality of accurate measuremenu usually facilitates 
objective consideration of the facts, and this leads to the acceptance 
and tmplementation of effective pohc.es When decs.ons are made 
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change be introduced generally, where applicable, so that all units 
will benefit and the organization derive maximum results from the 
change A demonstration of the value of testing changes expert 
mentally is often all that is required to convince an organization 
that usually it will derive greater benefits and derive them more 
rapidly if it tests the proposed changes experimental!) before intro- 
ducing them generally 


SOME ILLUSTRATIVE CASES 

In considering how to organize and conduct research devoted 
to the problems of a specific organization, it is important to dilTeren 
tiate two broadly different kinds of situations This is necessary 
since this difference affects the way in which the lesearch should 
be organized and conducted if effective use of the results is to be 
obtained One general type of situation is that in which the research 
deals with problems involved with the organization's internal opera 
tions Case A below is of this type In situations of this type the 
research operation is likely to have an appreciable impact on the 
organization and how it operates Consequently, it is extremely 
important to have the full support and interest of the verx top 
officers of the organization and to have their full participation in the 
plans for the research and its application Unless top management 
understands seeks, and fully supports the changes being undertaken, 
and understands the character of the resistance to the changes and 
the reasons for this resistance the attempts to apply the research 
will encounter real difficulty and may even fail 

The second general type of research situation is that in i\hich 
the research deals with problems faced by the organization which 
are outside of the organization Studies concerned with the buying 
behavior of consumers or with citizen response to governmental 
activities illustrate this general kind of problem Case B below is an 
example of this kind of research 


Case A (A Synthetic Case) 

The Director of Industnal Relations m Company A approached 
a research organization one day with the statement that the com 
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design included all the major hypotheses that should be encom 
passea in the study 

2 To inform all interested persons and groups about the study 
in a way that would enable them to ask questions about the study, 
to express any fears or reservations they might have, to discuss these 
fears and to obtain all the information they wanted or needed 
in order to have sufficient confidence in the study to cooperate 
fully with It 

3 To plan and conduct the study m such a way that all persons 
whose action was required to implement the study were fully in 
formed about it, were interested in it, and had an adequate oppor 
tunity to ask that the study obtain data they needed to make the 
best possible decisions on the problems falling within their areas 
of responsibility Taking sufficient time to secure this involvement 
was necessary for the results of the study to be implemented 

COMMENT It IS important to give all the people who are 
affected appreciably by a research project an opportunity to learn 
about it fully, to ask questions, and to make suggestions with regard 
to It Full access to information about the study and as much involve 
ment and participation as is feasible reduces fears of the research 
and increases the likelihood that the results will be used construe 
tively 

The planning sessions were not compressed into too short a 
period of time and were arranged in such a way that those persons 
whose actions would be required to implement the study had an 
opportunity to participate and to raise the questions that were on 
their minds One of the important functions of these planning 
sessions was to stimulate thinking to consider possible courses of 
action which might improve present and future operations This 
helped to suggest objectives for the research by indicating the data 
required to help in a choice among possible alternatite courses 
of action 

Another important function which these planning sessions were 
designed to perform was to help managerial and super\ isorj person 
nel to become better aware of the character and magnitude of some 
of the larger problems, especially human problems, which they faced 
Often people are so immersed in day to day operations that they 
are not aware of some of their larger and more important problems 
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As the plans for the study proceeded and 
the data got under way, it was felt desirabie to eep t e 
personnel, whose involvement was important, in orme 
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reporting back some of the preliminary resuits o t e r 
COMMENT Unless there are fa, rly frequent^J 
isled persons and groups on the progress f 

merest tends to lag Moreover, whenever anyone fe ' ^ 

vmnjormed, he tends to develop fearful and 
Consequently, whenever a research stag mformed of 

tons who have a relationship to a research p ] „^earch 

developments and progress, suspicious absence 

«« fostered These fearful and suspicious „ about 

ojoccurate information, are apt to lead to research can be 

‘'le research The egect of these rumors upon the r 

on 

In presenting the preliminary results, were re 

their preliminary and tentative nature mterpreta 

""nded that more detailed analysis might wel c g , j ^,^5 jq 
of the initial results An illustrauon that 


; the initial results An illustrauon uia ^ employees 

"jo the experience of a company which toward the 

0 'vere union members had a less favora ^ 


.x*a«.rs had a less hnue^er, ^^as 

'^“nipany than non union members This relations ’ 
inged as soon as the character ot \vork ol t toward 

'^ristant Blue collar workers had a less favora e union 

^company than white collar workers, and tv en ,^orkers and 

Ior"''x ”*"P analyzed separately for blue co tnembers 

'vhite collar workers, the findings were nititudes toward 

"ere found ,0 ha;e shghtly more favorable attitudes 

'^onipany than non union members officer to meas 
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supervisors and managers to develop the skills called for in their 
jobs There was continuous emphasis that the purpose of the re 
search v\as not to perform a policing function but rather to find 
what principles and methods are associated with better performance 
and how to help mediocre units to improve Throughout the study, 
the focus vsas on discovering the methods v^hich produce successful 
operation and learning how to train people to use these methods 
effectively It vsas important that supervisor) personnel not fear the 
research as a policing operation likely to hurt them but view it 
rather as a major effort to help each of them to learn how to perform 
his job more successful!) 

The results of the study in Company A fell into two distinct 
categories The first kind of results dealt with the measurements of 
compan) operations, either for the company as a whole or for sub- 
units of the compan) The other kind of data was based on cross 
tabulations which, for example, showed the different levels of 
emplo)ee products it) associated with different methods of super 
vision This second kind of analysis in addition to its value to Com 
pany A, adds to the body of fundamental knowledge about the 
principles of organizational functioning and leadership Each of 
these two kinds of results was reported to the company personnel 

In Company A, the results imijall) presented to company 
officers were of the first kind They vvere 'straight runs” showing 
results for the company as a whole and for some of the major operat 
mg departments of the company These and similar data were pre 
senied first to the President and the Executive Vice president in 
a meeting which was organized by the Director of Industrial Rela 
tions The presentation of the results was done by the member of the 
research organization who was in charge of the study 

This presentation did not include a detailed report presenting 
an analysis and an interpretation of the data On the contrary, 
only tabular results were presented and discussed The discussion 
and interpretation were participated m by all of those present at 
the meeting For example, questions were asked by the President 
as to why attitudes toward fi»^tline supervision differed so sub 
stantially m two departments Tentative hypotheses vvere offered 
b) him and the Executuc Vice president as to possible causes Tabu 
lations already completed by the research staff proied some ol these 
h)pothescs untenable and corroborated others Additional hjpoth 
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?aups of vice presidents had to do wit P about analyzing 
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group to approach these problems tn a constructive, problem solving 
orientation Employees attitudes and feelings came to be facts 
instead of things to be disregarded because they appeared to be too 
difficult to handle or did not clamor for immediate attention 

One significant change was introduced when the results were 
reported by each vice president to his immediate subordinates In the 
meetings with the President and the Executive Vice president, the 
results had been presented by the member of the research organiza 
tion who directed the study In the meetings conducted by each vice 
president for his staff, the responsibility for presenting the results 
was placed upon the vice president It was necessary for him to 
become sufficiently familiar with the methodology used in the study 
so that he could describe the methods used and answer any questions 
about the study which might arise during the presentation of the 
results For example, when some of the measurements indicated 
that a specific operation was not being done as well as might be 
expected, the people who were responsible for these operations often 
vvere likely to question the accuracy of the measurements In these 
circumstances, it was necessary for the vice president to be sufficiently 
well informed about the methods used and the magnitude and scope 
of possible errors so that he could answer these questions as to the 
probable accuracy of the results It was also necessary for him to 
point to other measurements and results which supported the specific 
result being questioned 

Each vice president was provided with tabulations showing the 
results for the company as a whole and for his department and for 
each division within his department Results were available also for 
other related or comparable departments so that each vice president 
could show his group how they compared with other comparable 
units within the company Data were also made available by the 
research organization showing measurements for comparable opera 
tions in other companies outside Company A All these results 
made it possible for the vice president to discuss with his staff the 
pattern of results for his department 

In each of these meetings conducted by a vice president, a mem 
er o t e staff of the Director of Industrial Relations was present 
to help answer any questions that might arise In addition, a member 
of the research organization was also present to answer any specific 
questions about the research which the vice president might not be 
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panv A was to analyze the data to discover the pattern o£ lelatiQ’’^ 
ships that existed between such variables as organizational stmctur 
and managerial and supervisory practices on the one hand ant ' 
employee productivity and job satisfaction on the other 

COMMENT Figures 2 and 3 show some of the relationship 
that were found and illustrate one of the methods of analysis used 
Such findings as these are not only of value to ‘^Company A but 
they, and generalizations based on them and on similar results, 
make an important contribution to our fundamental knowledge 
about organizational functioning and leadership (15, 16, 17, 18, 24) ^ 
The pattern of relationships obtained from this analysis was 
used by Company A in three major ways 

(1) These relationships were studied by the top officers of the 
company in staff meetings along with related data from similar 
studies The purpose of this analysis by top officers of the company 
was to examine present company policies in the light of these rela 
tionships and to consider minifying those policies where the results 
suggested that a modification would bring improvement All the 
different kinds of policies affecting employee behavior and reaction 
were studied, including policies dealing with such matteis as 
wages, promotion, job evaluation, company benefits training and 
supervisory practices 

(2) The training department built much of its maternl for 

training supervisors in human relations skills upon results obtained 
from the research They used the research results for case material 
and for pointing to some of the more common human relations 
problems that existed in the company In the training sessions super 
visors discussed what changes in supervisory practices and procedures 
would be most likely to bring about improvement, especially in rela 
xixuj So iJmsr* (rojb^eais wAicA, tAe data sAowea", occurred" 

most frequently They also discussed the patterns of relationships 
that were discovered to consider why certain kinds of supervisory 
behavior were associated with high or with low productivity or 
morale 

(3) The patterns of relationship which were discovered, along 
with the related material from other studies, were made aviihhle to 
all levels of management as they examined and studied the specific 
measurements dealing with their own operation The purpose of 
making these results available when the measurements v\ere being 
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of management after careful study and discussion of the research 
results, to introduce modifications m their operation which the per 
sonnel involved felt would be likely to bring about an improvement 
After these changes had been in operation for a sufficient period 
of time, top management authorized middle and lower levels of 
management to arrange with the research organization for whatever 
remeasurement was required in order to discover the effectiveness of 
the changes made 

This remeasurement, which was done in several different parts 
of the company, demonstrated that some of the changes introduced 
produced significant improvement and that other changes had a 
negligible effect Some situations required took two or three cycles 
of measurement, analysis, attempted change and remeasurement 
before improvement began to occur 


Case B (Research on a Population Being Served by an Agency) 

Case B represents another type of situation It differs from Case 
A m that the research is focused on persons outside the organization 
In both cases the results were used to bring about improvement in 
the operation of the organization In Case A, the changes involved 
internal relationships, in Case B the changes were program changes 
to help the organization achieve its objectives more effectively In 
both types of cases, those persons who are m positions of influence 
plauning the research and in decisions on 
research results 

are not likely to be applied 

culture ^ ^ Department of Agri 
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A^i^ltural War Boards m die Great Lakes states Lry afsistance 
possible m order that their campaign to increase mill production 
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not sure that research could help it or in what way The member<; 
felt that, by and large, they knew the situation in X State well and 
that their campaign plans were sound 

The X State Agricultural War Board believed the following 
to be the facts 

(1) The State Agricultural Statistician’s data for X State showed 
that the total number of cows in X State was larger than at anj 
previous time in the state s history Consequently, the War Board 
was of the opinion that virtually all the barns in X State were full 
and hence no further increase in number of cows was practicable 
at this time without an increase in barn space The members also 
knew that their own barns were full and that the barns of all the 
farmers they knew were full 

(2) They knew that the price of milk m relation to the price 
of feed made it highly profitable to feed milk cows heavily, includ 
ing grains and protein concentrates, in order to increase the milk 
production per cow They were following this practice themselves, 
and all the farmers they knew were following this practice The) 
assumed that this was true among farmers generally m X State 

Believing these to be the facts for X State, the War Board 
was of the opinion that the way to increase dairy production was 
{ ) to assure an adequate supply of feed grams and protein concen 
iraies at a reasonable price (2) to facilitate the building of addi 
tional barn space, and (3) to increase the available farm labor for 
daiy operation Consequently, the War Board proposed to ask that 
eed be made available at a speafied price and that priorities be 
panted or allocations made so that farmers could obtain all the lum 
er concrete, steel, plumbing, and other material required for 
bm dmg additional barn space The members also felt it desirable 
steps be taken to increase the farm labor supply ,n X State 
ith this background, the study was undertaken The major 
S "'“e producing 

to which could be taken 

o make possible a further increase in dairy production These objec 
lives included discovering what resources. 5^1 any. farmers felt the, 
needed to increase their dairy production Such resources were addi 
tional barn space milting machines other equipment, more feed 
higher quality feed, more labor, etc The study also had as an objec 
live discos ering the extent to which dair, farmers in X State s«re 
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tacted, the lack of bam spzce is not a determining factor in 
deciding whether or not to expand . . 

2 Labor supply in the state appears to be adequate, and 
the fear of a labor shortage does not seem to be an important 
obstacle to an expansion in production . . . 

3 Equipment needs appear to be no handicap to increased 
production. . . . 

4 An overwhelming proportion of X State farmers are satis 
ficd with the present price of milk . . . 

5 Running through all the data is the very noticeable thread 
that the farmer is uncertain, that he lacks confidence in the con 
tinuation of good prices, that he is apprehensive of a collapse 
of prices and markets after the war 

It appears that memories of the catastrophe that struck 
farmers after the last war and the experience of the recent 
hard, lean depression years base sensitized larmets to the danger 
of debt 

The great importance of guarantees of security and price 
over a long period of time is apparent . . . 

6 A third major kind of brake on expansion is a distinct 
lack of information First, farmers are confused and only 
partia y m ormed about the actions of the government in regard 
to production and prices Second, they lacl. InowJedge about the 
possibility of increased output through better feeding prac 

pansion of production in dairying can, of course, come 
in one or oth of tuo siays larger herds and better feeding The 
first insolses physicaf expans.on, uh.ch may be hampered by the 
.r, '=>>«• better feed 

IS nm t, ‘Respite uncertainties Hon eser, better feeding 

1. nor^the mean, by nhich the aserage farmer mil mabe large 

lam production are en 

Mm'el felt h I’ “'““belnnng proportion of those loter- 
le ed felt that they ,.e,e already at. or close to, the economic 
limit on feeding intensity. 

sho Jd M ""'"''■■P AAA comnt.ttcemen 

shotted a significantly different pattern from that obtained from 
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COMMENT The research staff deliberately made much use of 
the pattern of results that was found It pointed out that the town 
ship committeemen's altitudes and behavior were more like the 
results which the War Board had expected to find than were the 
attitudes and behavior of farmers generally This helped to show 
that informed farmers were behaving as the War Board believed all 
farmers were behaving The War Board members also recognized 
that their own friends and contacts tended, like themselves, to be 
much better informed than farmers generally This made it easy 
to suggest that probably, as the results indicated, many farmers were 
not so well informed as the War Board had previously assumed 

After several days of studying the findings and testing them by 
talking with farm leaders and groups of farmers, the Agricultural 
\ 'ar Board became convinced of the accuracy of the findings It then 
drastically revised its plans for the campaign to increase milk produc 
lion in X Slate The revised plans called, first, for a major informa 
tional campaign through the mass media, especially radio m which 
there would be much emphasis upon 

(1) the great increase in demand for milk and milk products 
the reasons for this increase and the reasons why this increase 
in demand ivould not collapse suddenly 

(2) the price support and guarantees to which the govern 
ment was committed through June 30 1943 

(3) the importance of increasing dairy production pnminly 
by the heavier feeding of grams and concentrates 


the plans ai?n" campaign through the mass media 

io^r“ou»'T ■" 
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and to answer their questions about the need for this mcrease The 
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ment, and the plan was modified and improved before more general 
application When the problem is a continuing one, further measure 
ments should be obtained ajter the general application 

The X State War Board found the assistance given to it m this 
study of so much value that the Agricultural War Boards of the other 
Great Lakes dairy states, when they learned of the study, asked 
for similar assistance in their own stales Time and financial limila 
lions made it impossible to conduct separate studies for the War 
Boards in each of the states but representatives of the Division of 
Program Sur\eys met with these War Boards and described the 
results of the X State study, the major conclusions which came from 
It, and the specific applications that were made Each of these State 
Agricultural War Boards then made corresponding applications in 
Its own state 

An indication of the value of this project can be obtained by 
examining the increase in dairy production that occurred in X 
Slate during the ensuing twelve months The increase m milk pro 
duction for the year November 1, 1941 to October SI, 1942 over 
the preceding ttselve month period was 6 7 percent This was one of 
highest rates of increase that occurred throughout the entire prewar 
and t\ar period Moreover, the study led the State War Board not 
to request the allocation of steel, lumber, cement, plumbing material, 
etc , which had originally been contemplated This conserved scarce 
material for urgent needs in the war effort 


Case C (A Story of What Not to Do) 

Case C, a condensed synthetic case, focuses on undesirable 
practices, reference to desirable practices has been largely omitted 
It is presented here because there continues to be evidence that, 
among business, government, and other agencies, attempts are 
made to use social science research in situations in which the re 
searchers are isolated from contact with operating personnel Such 
conditions present participation and involvement by operating per 
sonnel in the research process and prevent the research staff from 
dexeloping a full understanding of operating problems 

Tins organization was essentially a service research unit for 
operating go\ernmental agencies It was headed by an administrator 
who was competent but who happened to have no scientific training 
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obtain this information primarily by discussing then problems suf 
fictently with social scientists for the scientists to become familiar 
with the problems and to indicate what kinds of assistance research 
might be expected to contribute Consequently, in jiiuotionj such 
as this, in which the contacts between the operating personnel and 
the research staff are restricted to a minimum, there is little oppor 
tunity for the scientists to become familiar with operating problems 
and to suggest how research might be used to help solve them When 
the personnel of operating agencies are unable to obtain ideas and 
suggestions from the scientists, they necessarily limit their requests 
for research only to the very few possibilities which their restricted 
knowledge of research and its potentiality suggests 

Under such conditions, it is virtually impossible for the operat 
mg personnel to grasp the full magnitude of what research can do 
for them and to seek this assistance Similarly, the research person 
nel without knowledge of the operating problems, do not recognize 
the need that operating personnel have for research and so are 
unable to suggest possible research projects In these circumstances, 
research is done only on minor aspects of problems or on superficial 
problems 

After a request for research had been given the research staff, 
a study design was developed The study design was then reviewed by 
one or more members of a group of research consultants which this 
administrator had appointed to advise him These consultants 
usually spent only one or two days a week with the organization 
and therefore had only a cursory knowledge of the problems of the 
agency and consequently of the required research Their discussions 
and advice usually resulted in some modifications in the research 
design After these changes the consultants generally gave their 
approval which in turn resulted in the administrators authorizing 
the staff to proceed with the research project 

COMMENT Whenever research plans are reviewed by out 
side experts who do not have suffUcient time to become thoroughly 
familiar with the problem, the research staff generally feels under 
pressure to use a research design which will be subject to a minimum 
of criticism and change This, of course, results in a tendency to 
use a design of a traditional character rather than one which is 
specifically and efficiently focused on the problem 
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obtain this information primarily by discussing their problems suf 
ficiently with social scientists for the scientists to become familiar 
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tumty for the scientists to become familiar with operating problems 
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the personnel of operating agencies are unable to obtain ideas and 
suggestions from the scientists, they necessarily limit their requests 
for research only to the very few possibilities which their restricted 
knowledge of research and its potentiality suggests 

Under such conditions, it is virtually impossible for the operat 
mg personnel to grasp the full magnitude of what research can do 
for them and to seek this assistance Similarly, the research person 
nel, without knowledge of the operating problems, do not recognize 
the need that operating personnel have for research and so are 
unable to suggest possible research projects In these circumstances, 
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the original analysis. Frequently, however, the original data could 
be retabulated or analyzed further so as to yield significant facts 
to help answer the new questions. But the liaison person was unable 
to suggest that this could be done because he v\ras unfamiliar with 
the methodology and the data. 

COMMENT: If research for an operating agency is done xuell, 
it will yield generalizations applicable to many problems and policies 
and not just specific answers to specific questions. But if these gen- 
eralizations are to be used fully and correctly, it is essential that 
they be understood by the operating personnel. An effective under- 
standing of the generalizations usually is best acquired by discussion 
of their implications for specific operating problems. This requires 
discussions of such problems by key operating persons with scientists 
who either have conducted the research or know it well. When 
administrative regulations prevent such discussions, the use of the 
research results jf seriously limited. 

There w another kind of problem which occurs when a liaisor 
person without research training is placed between the research 
and the operating personnel. Often the most intelligent application 
of the research findings involves recognizing the presence of patteini 
of results and not merely considering individual items. Moieover 
these pattern interpretations may require knowledge not only of the 
findings from this current study but knowledge of the results of 
other research, including work done previously in this agency and 
xvork done elsewhere and published. The scientists who conduct th^ 
research have this background knowledge of other research results 
A liaison person who lacks scientific training almost never has this 
knowledge and consequently is less likely to be able to help the 
operating person to see the patterns of results that have relevance 
for their decisions. 

As might be expected, operating agencies found that research 
conducted in these circumstances was not very useful to them. 
Progressively they asked for and used research less and less. 

This research organization was finally abolished. Some parts 
of it, however, were shifted to other agencies. Some of these, aftei 
they were shifted, managed to develop a close, functional relation 
ship with the operating agencies whose problems they were studying 
These parts survived and performed important and valuable reseaich 
functions. 
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